A New Generalised Inverse Polynomial Model in the Exploration of Response Surface Methodology

Size: px
Start display at page:

Download "A New Generalised Inverse Polynomial Model in the Exploration of Response Surface Methodology"

Transcription

1 Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): Scholarlink Research Institute Journals 011 (ISSN: ) jeteas.scholarlinkresearch.org Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): (ISSN: ) A New Generalised Inverse Polynomial Model in the Exploration of Response Surface Methodology 1 Owoseni D.O. Fatokun J.O 1 Department of Mathematics Statistics Federal Polytechnic Ado-Ekiti Ekiti State Nigeria Department of Mathematical Sciences Nasarawa State University Keffi Nigeria Corresponding Author: Fatokun J.O Abstract This paper concerns the application of some theories surrounding the exploration of response surface using inverse polynomial models which include the construction of response surface designs the methods of parameter estimation by the iterative methods of non-linear least square the method of optimization such as the Fletcher-Reeves conjugate Gradient method. We verify the adequacy of the model using a well analysed experimental data on maize-bush beans intercrop to determine the appropriate plant densities which yield optimum responses. This new approach compares favourably for a general surface (K=k P=p factors levels respectively) Keywords: response surface methodology (RSM) conjugate gradient method inverse polynomial l equivalent ratio (LER) INTRODUCTION Response surface methodology (RSM) is a set of statistical allied procedures originally developed by Box Wilson (1951) its two main components being the design of experiments regression analysis. The methodology is applied to study the yield or output of a system as it varies in response to the changing levels of one or more input or design variables.the experimental aspect deals with the choice of variables their various levels regression analyses enables a mathematical model to be fitted to the varying yield this model is then investigated. Until recently when response surface were explored via inverse polynomials Box Hunter (1957) the ordinary polynomials the second order in particular have been extensively employed in exploring response surfaces. This is because it is generally accepted that it is computationally simple to work with easy to locate the optimum response. OBJECTIVES OF THE STUDY The main objective of the study is to develop a generalised model for the Response Surface Method using inverse polynomial approach. R. S. M. was developed mainly with a view towards industrial experimentation production (Box Wilson 1951) but it has found application also in agriculture (Mead Pike 1975) in medical settings (Carter Wampler Stablein 1983) more recently in connection with off-line quality control (Vining Myers 1990) And even though R S M has proven to be useful in practice it suffers a serious defect namely that the form of the response surface depends on the choice of units for the input variables To illustrate this point we consider a simple example. The relationship Y=x 1 + x can be pictured as a two-dimensional surface in a three-dimensional space giving the dependence of Y on X 1 X. If we change the units for the input variables to X * 1 = X 1 X * = 3X then the relationship becomes Y= * * 1/4X 1 + 1/9 X The Inverse Polynomial Model: Derivation If represent the levels of k experimental factors Y is the mean response then the inverse polynomial response function is defined by (1) We write (1) as where = () of factors of levels of factor i (3) 1059

2 Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): (ISSN: ) If we define = for k=1 p= we have from (3) (4) The inverse polynomial model (3) were first used for yield density relationship for agricultural crops where is defined as the yield per plant. Generalised Non-Linear Polynomial Models Assuming values for some parameters in (3) results into some of the members of non-linear polynomial models see Nduka (1994). (i) Planer Surfaces (K P ;k=1 P=) This surface is a case of a single factor experiment at levels. The following are the family members (5) Equation (5) is the Miitshelich model as in Nelder (196). This model was widely applied in the study of fertization of soil. Miitshelich considered c to be a constant however later studies showed that it varies. Fletcher R Reeves C.M. (1964) Atkinson (1969) Draper (1971) generalized Miitshelich model to the family of exponential regression as (6) The reciprocal of (7) yields This model is widely used by demographers for generating life tables by economist for predicting From (1) if we let = i.e. = price changes in time series for studying trend while (8) is a form of logistic model used in the study. We have of growth curves probit analysis. In this study for =1 (ii) becomes following the same approach manipulations we generate for planer quadratic surfaces as follows. However if we let =1 in in the table below: (ii) we have. If 01 ; CD A l e ; 11 Ae y we have Table 1: Generalized Models fork-level p-factors surfaces. Surface Type Factor(k) Level(p) Model Planer (7) (8) = ( 0 < < 1 ) Planer ( d1dd3) 001( d1d ) 010 ( d1d3) 011d1 Quadratic 1 3 ( d d ) d d Quadratic 3 General K P d1d d d d d d d d d d d d d NUMERICAL EXPERIMENTATION We apply the above generalized model to the following agricultural problem. Field experiments were planted in two seasons.in the first season attempted maize densities were plants per beans densities were plants per. All possible combinations give 48 density treatments. In the second season number of maize densities were reduced to five ( ) plants per number of beans densities to five (08164m 3 plants per ).Maize were planted in the middle of bed raised about 15cm above the furrows with 1m from centre to centre of the beds. Beans were intercropped in two rows / one row on either side 15cm from the maize planted on the same day. Both crops were over seeded thinned at 3 weeks after planting to desire densities In each season the design was a split-plot with maize densities as main plot beans densities as split-

3 Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): (ISSN: ) plot. Each bean density plot was 6mlong beds (m) plots had end borders of 0.5m 1m in the first second season respectively. There were 4 replicates in each experiment. The L Equivalent Ratio (LER) is given by the formula LER i 1 yi ( I ) yi (M ) where design model is not a good a fit thus the need for the second order design which is formulated as follows: Hence for a rotatable orthogonal composite second order design we must have a total of (9) is the intercrop yield of the experimental runs. crop is the monocrop of the crop. Using (9) the LER for Maize-Beans intercrop is given in the table below: Table : LER for Maize-Beans intercrop Beans density plants/m Maize density plants /m The data was further treated as pseudo-factorial design whereby maize beans densities were the two factors their levels taken as the four levels of the two factors. First Order Design The response surface design is presented below Let be the factor levels each factor has four levels Hence : The models (i) (ii) are fitted into the data on monoculture maize using non-linear (iterative) least square method of Gauss-Newton Maarquardt. Below is the summary of the results : The variable D for the first order response surface is decoded as follows Table 3: Summary of Results of Non-Linear Model using Gauss-Newton Method Model parameters estimated iteration Residual mean Sq. Mitshelich Inverse Polynomial _ _ Table 4: Summary of Results of Non-Linear Model using Marquardt Method Optimum Response for Monoculture Planting System Note that that the design is orthogonal.. This implies Second Order Design From the plot of response curve for the first order design in (Owoseni 1997) we assume that first order 1061 Model parameters estimated iteration Residual mean Sq. Mitshelich Inverse Polynomial

4 Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): (ISSN: ) The response model is obtained as After iterations as (10) is a descent method we have We desire to obtain the design points to conduct the experiment to give the optimum yield. Hence we have M in f (X ) X 1 also for all (11) iteration. 1 Using the Secant Technique of minimizing the unidimensional regression model we have we have the table below Table 5: Summary of Result using the Secant Technique ) hence we stop DISCUSSION OF RESULTS When plant densities are increased indefinitely this leads to overcrowding by limit theorem as the variable tends to infinity the yield will greatly be reduced as nutrient absorption which is a function of growth is reduced as a result of competition from neighboring plants. Also when plants densities are reduced this leads to isolated plants by limits theorem the yield is also effectively reduced because such plants will not absorb enough nutrient as a result of poor root contact. We choose arbitrary stationary points Iteration since C.G.M Gauss-Newton Marquardt methods of non-linear parameter estimation considered gave approximately the same result except for the number of iterations which was consistently fewer using Gauss-Newton a default method in the Statistical software package (SAS). Since the optimum yield in any plant-yield relationship requires an optimum plant setting (densities) therefore we need the combination of Secant Conjugate Gradient method (FletcherReeves algorithm) techniques for monoculture inter-cropping situations respectively. An optimum of were found to give rise to an optimum maize yield of. And for intercropping of maize with beans optimum plant densities of respectively gave rise to optimum yield of.this reveals the advantage of maizebeans intercrop over maize monoculture which supports the empirical findings of Francis et al [1983]. Note that Therefore stop iteration.hence i.e. The expected minimum yield is therefore when an optimum plant density of is used. Optimum Response for Intercrop Planting System The response model is obtained as To obtain the optimum response we find the optimum design points to conduct the experiment as follows: Using the Conjugate Gradient Technique (FletcherReeves algorithm) to determine the design points which gives the optimum yield. Let setting the stationary points as CONCLUSION We have presented a generalised inverse polynomial model for response surfaces applied to an intercrop problem. We verify the adequacy of the model using a well analysed experimental data on maize-bush beans intercrop to determine the appropriate plant densities which yield optimum responses. This new approach compares favourably for a general surface (K=k P=p factors levels respectively)..these are arbitrary values which represent the densities of the two intercropped plants maize beans respectively. The results are given below: Table 6: Summary Of Result using the C.G.M.(Fetcher-Reeves Algorithm) Iteration REFERENCES Atkinson A.C. (1969) Constrained minimization the Design of Experiments Technimetrics

5 Journal of Emerging Trends in Engineering Applied Sciences (JETEAS) (6): (ISSN: ) Berness J.P (1965) An algorithm for solving nonlinear equations based on the Secant method Comp. Journal Berry G. (1967) A mathematical model relating plant yield with arrangement for regularly spaced crops Biometrics BleadaleJ.K. Nelder (1960) Plant population yield Nature London Box (1954) The exploration exploitation of Response surfaces; some general considerations examples Biometrics Nelder J.A. (1966) Inverse polynomials a useful group of multi-factor response functions. Biometrics Owoseni O.O. (1997) Inverse polynomials in the Exploration of Response Surfaces A Master of Sc. Thesis University of Ibadan. Samuel H.S. Ray M. (1961) Optimum estimation of Gradient Direction in Steepest Ascent experiments Biometrics Box Draper (1959) A b asis for the selection of a response surface design J.A.S.A Box M.T. (1969) Planning experiment to test the adequacy of non-linear models Appl. Stat Box Hunter (1957) Multifactor Designs for exploring response surfaces. Ann. Math. Stat Box Wilson (1951). On the experimental attainment of optimum conditions (with discussion) J.R. Statistics soc. B Cronoder (1987). On linear Quadratic estimating function Biometrika Draper (1971): On Lack of fit Technometrics Draper (1986) Response Surface design in flexible regions J.A.S.A Edmonson (1991) Agricultural Response Surface experiment based on four-level factorial. Designs Biometrics HestenesM.R. (1980) Conjugate Direction Methods in optimization Springer-Verlay New York. Fletcher R Reeves C.M. (1964) Functional minimization by Conjugate Gradient The comp. J FrancisC.H. Prager Teteder (198) Density interaction in Tropical inter-cropping maize Bush beans field crop Research Nelder J.A. (196) New kid of systematic Design for spacing experiments Biometrics

Optimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23

Optimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23 Optimization: Nonlinear Optimization without Constraints Nonlinear Optimization without Constraints 1 / 23 Nonlinear optimization without constraints Unconstrained minimization min x f(x) where f(x) is

More information

OPTIMIZATION OF FIRST ORDER MODELS

OPTIMIZATION OF FIRST ORDER MODELS Chapter 2 OPTIMIZATION OF FIRST ORDER MODELS One should not multiply explanations and causes unless it is strictly necessary William of Bakersville in Umberto Eco s In the Name of the Rose 1 In Response

More information

7. Response Surface Methodology (Ch.10. Regression Modeling Ch. 11. Response Surface Methodology)

7. Response Surface Methodology (Ch.10. Regression Modeling Ch. 11. Response Surface Methodology) 7. Response Surface Methodology (Ch.10. Regression Modeling Ch. 11. Response Surface Methodology) Hae-Jin Choi School of Mechanical Engineering, Chung-Ang University 1 Introduction Response surface methodology,

More information

Unconstrained Multivariate Optimization

Unconstrained Multivariate Optimization Unconstrained Multivariate Optimization Multivariate optimization means optimization of a scalar function of a several variables: and has the general form: y = () min ( ) where () is a nonlinear scalar-valued

More information

Artificial Neural Networks. MGS Lecture 2

Artificial Neural Networks. MGS Lecture 2 Artificial Neural Networks MGS 2018 - Lecture 2 OVERVIEW Biological Neural Networks Cell Topology: Input, Output, and Hidden Layers Functional description Cost functions Training ANNs Back-Propagation

More information

Statistical Modeling for Citrus Yield in Pakistan

Statistical Modeling for Citrus Yield in Pakistan European Journal of Scientific Research ISSN 1450-216X Vol.31 No.1 (2009, pp. 52-58 EuroJournals Publishing, Inc. 2009 http://www.eurojournals.com/ejsr.htm Statistical Modeling for Citrus Yield in Pakistan

More information

Optimization. Totally not complete this is...don't use it yet...

Optimization. Totally not complete this is...don't use it yet... Optimization Totally not complete this is...don't use it yet... Bisection? Doing a root method is akin to doing a optimization method, but bi-section would not be an effective method - can detect sign

More information

ON D-OPTIMAL DESIGNS FOR ESTIMATING SLOPE

ON D-OPTIMAL DESIGNS FOR ESTIMATING SLOPE Sankhyā : The Indian Journal of Statistics 999, Volume 6, Series B, Pt. 3, pp. 488 495 ON D-OPTIMAL DESIGNS FOR ESTIMATING SLOPE By S. HUDA and A.A. AL-SHIHA King Saud University, Riyadh, Saudi Arabia

More information

On Application Of Modified Gradient To Extended Conjugate Gradient Method Algorithm For Solving Optimal Control Problems

On Application Of Modified Gradient To Extended Conjugate Gradient Method Algorithm For Solving Optimal Control Problems IOSR Journal of Mathematics (IOSR-JM) e-issn: 2278-5728, p-issn:2319-765x. Volume 9, Issue 5 (Jan. 2014), PP 30-35 On Application Of Modified Gradient To Extended Conjugate Gradient Method Algorithm For

More information

Response Surface Methodology

Response Surface Methodology Response Surface Methodology Process and Product Optimization Using Designed Experiments Second Edition RAYMOND H. MYERS Virginia Polytechnic Institute and State University DOUGLAS C. MONTGOMERY Arizona

More information

Response Surface Methodology IV

Response Surface Methodology IV LECTURE 8 Response Surface Methodology IV 1. Bias and Variance If y x is the response of the system at the point x, or in short hand, y x = f (x), then we can write η x = E(y x ). This is the true, and

More information

Optimization and Root Finding. Kurt Hornik

Optimization and Root Finding. Kurt Hornik Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding

More information

Response Surface Methodology:

Response Surface Methodology: Response Surface Methodology: Process and Product Optimization Using Designed Experiments RAYMOND H. MYERS Virginia Polytechnic Institute and State University DOUGLAS C. MONTGOMERY Arizona State University

More information

Numerical Schemes Based on Non-Standard Methods for Initial Value Problems in Ordinary Differential Equations

Numerical Schemes Based on Non-Standard Methods for Initial Value Problems in Ordinary Differential Equations Journal of Emerging Trends in Engineering and Applied Sciences (JETEAS) 3 (): 49-55 Scholarlink Research Institute Journals, 22 (ISSN: 24-76) jeteas.scholarlinkresearch.org Numerical Schemes Based on Non-Standard

More information

Non-linear least squares

Non-linear least squares Non-linear least squares Concept of non-linear least squares We have extensively studied linear least squares or linear regression. We see that there is a unique regression line that can be determined

More information

RESPONSE SURFACE MODELLING, RSM

RESPONSE SURFACE MODELLING, RSM CHEM-E3205 BIOPROCESS OPTIMIZATION AND SIMULATION LECTURE 3 RESPONSE SURFACE MODELLING, RSM Tool for process optimization HISTORY Statistical experimental design pioneering work R.A. Fisher in 1925: Statistical

More information

Numerical Analysis of Electromagnetic Fields

Numerical Analysis of Electromagnetic Fields Pei-bai Zhou Numerical Analysis of Electromagnetic Fields With 157 Figures Springer-Verlag Berlin Heidelberg New York London Paris Tokyo Hong Kong Barcelona Budapest Contents Part 1 Universal Concepts

More information

FIVE-FACTOR CENTRAL COMPOSITE DESIGNS ROBUST TO A PAIR OF MISSING OBSERVATIONS. Munir Akhtar Vice-Chanselor, Islamia University Bahawalpur, Pakistan.

FIVE-FACTOR CENTRAL COMPOSITE DESIGNS ROBUST TO A PAIR OF MISSING OBSERVATIONS. Munir Akhtar Vice-Chanselor, Islamia University Bahawalpur, Pakistan. Journal of Research (Science), Bahauddin Zakariya University, Multan, Pakistan. Vol.2, No.2, December 2, pp. 5-5 ISSN 2-2 FIVE-FACTOR CENTRAL COMPOSITE DESIGNS ROBUST TO A PAIR OF MISSING OBSERVATIONS

More information

Methods that avoid calculating the Hessian. Nonlinear Optimization; Steepest Descent, Quasi-Newton. Steepest Descent

Methods that avoid calculating the Hessian. Nonlinear Optimization; Steepest Descent, Quasi-Newton. Steepest Descent Nonlinear Optimization Steepest Descent and Niclas Börlin Department of Computing Science Umeå University niclas.borlin@cs.umu.se A disadvantage with the Newton method is that the Hessian has to be derived

More information

DESIGN OF EXPERIMENT ERT 427 Response Surface Methodology (RSM) Miss Hanna Ilyani Zulhaimi

DESIGN OF EXPERIMENT ERT 427 Response Surface Methodology (RSM) Miss Hanna Ilyani Zulhaimi + DESIGN OF EXPERIMENT ERT 427 Response Surface Methodology (RSM) Miss Hanna Ilyani Zulhaimi + Outline n Definition of Response Surface Methodology n Method of Steepest Ascent n Second-Order Response Surface

More information

Contents. Preface. 1 Introduction Optimization view on mathematical models NLP models, black-box versus explicit expression 3

Contents. Preface. 1 Introduction Optimization view on mathematical models NLP models, black-box versus explicit expression 3 Contents Preface ix 1 Introduction 1 1.1 Optimization view on mathematical models 1 1.2 NLP models, black-box versus explicit expression 3 2 Mathematical modeling, cases 7 2.1 Introduction 7 2.2 Enclosing

More information

Regression, Curve Fitting and Optimisation

Regression, Curve Fitting and Optimisation Supervised by Elena Zanini STOR-i, University of Lancaster 4 September 2015 1 Introduction Root Finding 2 3 Simulated Annealing 4 5 The Rosenbrock Banana Function 6 7 Given a set of data, what is the optimum

More information

Classes of Second-Order Split-Plot Designs

Classes of Second-Order Split-Plot Designs Classes of Second-Order Split-Plot Designs DATAWorks 2018 Springfield, VA Luis A. Cortés, Ph.D. The MITRE Corporation James R. Simpson, Ph.D. JK Analytics, Inc. Peter Parker, Ph.D. NASA 22 March 2018 Outline

More information

Vasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks

Vasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks C.M. Bishop s PRML: Chapter 5; Neural Networks Introduction The aim is, as before, to find useful decompositions of the target variable; t(x) = y(x, w) + ɛ(x) (3.7) t(x n ) and x n are the observations,

More information

2 Introduction to Response Surface Methodology

2 Introduction to Response Surface Methodology 2 Introduction to Response Surface Methodology 2.1 Goals of Response Surface Methods The experimenter is often interested in 1. Finding a suitable approximating function for the purpose of predicting a

More information

Statistics 580 Optimization Methods

Statistics 580 Optimization Methods Statistics 580 Optimization Methods Introduction Let fx be a given real-valued function on R p. The general optimization problem is to find an x ɛ R p at which fx attain a maximum or a minimum. It is of

More information

Response Surface Methods

Response Surface Methods Response Surface Methods 3.12.2014 Goals of Today s Lecture See how a sequence of experiments can be performed to optimize a response variable. Understand the difference between first-order and second-order

More information

Numerical Optimization

Numerical Optimization Numerical Optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Spring 2010 Emo Todorov (UW) AMATH/CSE 579, Spring 2010 Lecture 9 1 / 8 Gradient descent

More information

Programming, numerics and optimization

Programming, numerics and optimization Programming, numerics and optimization Lecture C-3: Unconstrained optimization II Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428

More information

Nonlinear Regression. Summary. Sample StatFolio: nonlinear reg.sgp

Nonlinear Regression. Summary. Sample StatFolio: nonlinear reg.sgp Nonlinear Regression Summary... 1 Analysis Summary... 4 Plot of Fitted Model... 6 Response Surface Plots... 7 Analysis Options... 10 Reports... 11 Correlation Matrix... 12 Observed versus Predicted...

More information

Principal Component Analysis, an Aid to Interpretation of Data. A Case Study of Oil Palm (Elaeis guineensis Jacq.)

Principal Component Analysis, an Aid to Interpretation of Data. A Case Study of Oil Palm (Elaeis guineensis Jacq.) Journal of Emerging Trends in Engineering and Applied Sciences (JETEAS) 4(2): 237-241 Scholarlink Research Institute Journals, 2013 (ISSN: 2141-7016) jeteas.scholarlinkresearch.org Journal of Emerging

More information

Chapter 10 Conjugate Direction Methods

Chapter 10 Conjugate Direction Methods Chapter 10 Conjugate Direction Methods An Introduction to Optimization Spring, 2012 1 Wei-Ta Chu 2012/4/13 Introduction Conjugate direction methods can be viewed as being intermediate between the method

More information

Motivation: We have already seen an example of a system of nonlinear equations when we studied Gaussian integration (p.8 of integration notes)

Motivation: We have already seen an example of a system of nonlinear equations when we studied Gaussian integration (p.8 of integration notes) AMSC/CMSC 460 Computational Methods, Fall 2007 UNIT 5: Nonlinear Equations Dianne P. O Leary c 2001, 2002, 2007 Solving Nonlinear Equations and Optimization Problems Read Chapter 8. Skip Section 8.1.1.

More information

Constrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method.

Constrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method. Optimization Unconstrained optimization One-dimensional Multi-dimensional Newton s method Basic Newton Gauss- Newton Quasi- Newton Descent methods Gradient descent Conjugate gradient Constrained optimization

More information

FALL 2018 MATH 4211/6211 Optimization Homework 4

FALL 2018 MATH 4211/6211 Optimization Homework 4 FALL 2018 MATH 4211/6211 Optimization Homework 4 This homework assignment is open to textbook, reference books, slides, and online resources, excluding any direct solution to the problem (such as solution

More information

Comparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming Problems

Comparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming Problems International Journal of Scientific and Research Publications, Volume 3, Issue 10, October 013 1 ISSN 50-3153 Comparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming

More information

Optimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections

Optimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations Subsections One-dimensional Unconstrained Optimization Golden-Section Search Quadratic Interpolation Newton's

More information

MATH 4211/6211 Optimization Basics of Optimization Problems

MATH 4211/6211 Optimization Basics of Optimization Problems MATH 4211/6211 Optimization Basics of Optimization Problems Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 A standard minimization

More information

Machine Learning and Data Mining. Linear regression. Kalev Kask

Machine Learning and Data Mining. Linear regression. Kalev Kask Machine Learning and Data Mining Linear regression Kalev Kask Supervised learning Notation Features x Targets y Predictions ŷ Parameters q Learning algorithm Program ( Learner ) Change q Improve performance

More information

Chemometrics Unit 4 Response Surface Methodology

Chemometrics Unit 4 Response Surface Methodology Chemometrics Unit 4 Response Surface Methodology Chemometrics Unit 4. Response Surface Methodology In Unit 3 the first two phases of experimental design - definition and screening - were discussed. In

More information

NUMERICAL COMPARISON OF LINE SEARCH CRITERIA IN NONLINEAR CONJUGATE GRADIENT ALGORITHMS

NUMERICAL COMPARISON OF LINE SEARCH CRITERIA IN NONLINEAR CONJUGATE GRADIENT ALGORITHMS NUMERICAL COMPARISON OF LINE SEARCH CRITERIA IN NONLINEAR CONJUGATE GRADIENT ALGORITHMS Adeleke O. J. Department of Computer and Information Science/Mathematics Covenat University, Ota. Nigeria. Aderemi

More information

PRODUCT QUALITY IMPROVEMENT THROUGH RESPONSE SURFACE METHODOLOGY : A CASE STUDY

PRODUCT QUALITY IMPROVEMENT THROUGH RESPONSE SURFACE METHODOLOGY : A CASE STUDY PRODUCT QULITY IMPROVEMENT THROUGH RESPONSE SURFCE METHODOLOGY : CSE STUDY HE Zhen, College of Management and Economics, Tianjin University, China, zhhe@tju.edu.cn, Tel: +86-22-8740783 ZHNG Xu-tao, College

More information

Efficient Choice of Biasing Constant. for Ridge Regression

Efficient Choice of Biasing Constant. for Ridge Regression Int. J. Contemp. Math. Sciences, Vol. 3, 008, no., 57-536 Efficient Choice of Biasing Constant for Ridge Regression Sona Mardikyan* and Eyüp Çetin Department of Management Information Systems, School of

More information

Generalized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science.

Generalized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science. Texts in Statistical Science Generalized Linear Mixed Models Modern Concepts, Methods and Applications Walter W. Stroup CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint

More information

NUMERICAL MATHEMATICS AND COMPUTING

NUMERICAL MATHEMATICS AND COMPUTING NUMERICAL MATHEMATICS AND COMPUTING Fourth Edition Ward Cheney David Kincaid The University of Texas at Austin 9 Brooks/Cole Publishing Company I(T)P An International Thomson Publishing Company Pacific

More information

Lecture 10: September 26

Lecture 10: September 26 0-725: Optimization Fall 202 Lecture 0: September 26 Lecturer: Barnabas Poczos/Ryan Tibshirani Scribes: Yipei Wang, Zhiguang Huo Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These

More information

Laboratory Exercise System Identification. Laboratory Experiment AS-PA6. "Design of Experiments"

Laboratory Exercise System Identification. Laboratory Experiment AS-PA6. Design of Experiments Formel-Kapitel Abschnitt FACHGEBIET Systemanalyse Laboratory Exercise System Identification Laboratory Experiment AS-PA6 "Design of Experiments" Responsible professor: Dr.-Ing. Thomas Glotzbach Responsible

More information

A three point formula for finding roots of equations by the method of least squares

A three point formula for finding roots of equations by the method of least squares Journal of Applied Mathematics and Bioinformatics, vol.2, no. 3, 2012, 213-233 ISSN: 1792-6602(print), 1792-6939(online) Scienpress Ltd, 2012 A three point formula for finding roots of equations by the

More information

Notes on Some Methods for Solving Linear Systems

Notes on Some Methods for Solving Linear Systems Notes on Some Methods for Solving Linear Systems Dianne P. O Leary, 1983 and 1999 and 2007 September 25, 2007 When the matrix A is symmetric and positive definite, we have a whole new class of algorithms

More information

Appendix IV Experimental Design

Appendix IV Experimental Design Experimental Design The aim of pharmaceutical formulation and development is to develop an acceptable pharmaceutical formulation in the shortest possible time, using minimum number of working hours and

More information

Nonlinear Optimization: What s important?

Nonlinear Optimization: What s important? Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global

More information

Optimization Methods

Optimization Methods Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available

More information

Chapter 4. Unconstrained optimization

Chapter 4. Unconstrained optimization Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file

More information

Linear Models for Regression. Sargur Srihari

Linear Models for Regression. Sargur Srihari Linear Models for Regression Sargur srihari@cedar.buffalo.edu 1 Topics in Linear Regression What is regression? Polynomial Curve Fitting with Scalar input Linear Basis Function Models Maximum Likelihood

More information

ON THE DESIGN POINTS FOR A ROTATABLE ORTHOGONAL CENTRAL COMPOSITE DESIGN

ON THE DESIGN POINTS FOR A ROTATABLE ORTHOGONAL CENTRAL COMPOSITE DESIGN ON THE DESIGN POINTS FOR A ROTATABLE ORTHOGONAL CENTRAL COMPOSITE DESIGN Authors: CHRISTOS P. KITSOS Department of Informatics, Technological Educational Institute of Athens, Greece (xkitsos@teiath.gr)

More information

Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd Edition

Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd Edition Brochure More information from http://www.researchandmarkets.com/reports/705963/ Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd Edition Description: Identifying

More information

D-optimal Designs for Factorial Experiments under Generalized Linear Models

D-optimal Designs for Factorial Experiments under Generalized Linear Models D-optimal Designs for Factorial Experiments under Generalized Linear Models Jie Yang Department of Mathematics, Statistics, and Computer Science University of Illinois at Chicago Joint research with Abhyuday

More information

EAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science

EAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one

More information

Inverse Problems and Optimal Design in Electricity and Magnetism

Inverse Problems and Optimal Design in Electricity and Magnetism Inverse Problems and Optimal Design in Electricity and Magnetism P. Neittaanmäki Department of Mathematics, University of Jyväskylä M. Rudnicki Institute of Electrical Engineering, Warsaw and A. Savini

More information

Optimization II: Unconstrained Multivariable

Optimization II: Unconstrained Multivariable Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Optimization II: Unconstrained Multivariable 1

More information

Conjugate Directions for Stochastic Gradient Descent

Conjugate Directions for Stochastic Gradient Descent Conjugate Directions for Stochastic Gradient Descent Nicol N Schraudolph Thore Graepel Institute of Computational Science ETH Zürich, Switzerland {schraudo,graepel}@infethzch Abstract The method of conjugate

More information

, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are

, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues

More information

Conjugate Gradient (CG) Method

Conjugate Gradient (CG) Method Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

More information

Mixture Designs Based On Hadamard Matrices

Mixture Designs Based On Hadamard Matrices Statistics and Applications {ISSN 2452-7395 (online)} Volume 16 Nos. 2, 2018 (New Series), pp 77-87 Mixture Designs Based On Hadamard Matrices Poonam Singh 1, Vandana Sarin 2 and Rashmi Goel 2 1 Department

More information

Linear Models in Machine Learning

Linear Models in Machine Learning CS540 Intro to AI Linear Models in Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu We briefly go over two linear models frequently used in machine learning: linear regression for, well, regression,

More information

A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL

A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Discussiones Mathematicae Probability and Statistics 36 206 43 5 doi:0.75/dmps.80 A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Tadeusz Bednarski Wroclaw University e-mail: t.bednarski@prawo.uni.wroc.pl

More information

Review. Numerical Methods Lecture 22. Prof. Jinbo Bi CSE, UConn

Review. Numerical Methods Lecture 22. Prof. Jinbo Bi CSE, UConn Review Taylor Series and Error Analysis Roots of Equations Linear Algebraic Equations Optimization Numerical Differentiation and Integration Ordinary Differential Equations Partial Differential Equations

More information

DESIGN AND ANALYSIS OF EXPERIMENTS Third Edition

DESIGN AND ANALYSIS OF EXPERIMENTS Third Edition DESIGN AND ANALYSIS OF EXPERIMENTS Third Edition Douglas C. Montgomery ARIZONA STATE UNIVERSITY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore Contents Chapter 1. Introduction 1-1 What

More information

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Methods in Geochemistry and Geophysics, 36 GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Michael S. ZHDANOV University of Utah Salt Lake City UTAH, U.S.A. 2OO2 ELSEVIER Amsterdam - Boston - London

More information

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Cheng Soon Ong & Christian Walder. Canberra February June 2018 Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression

More information

Generalized Linear Models

Generalized Linear Models Generalized Linear Models Advanced Methods for Data Analysis (36-402/36-608 Spring 2014 1 Generalized linear models 1.1 Introduction: two regressions So far we ve seen two canonical settings for regression.

More information

Chapter 6: Derivative-Based. optimization 1

Chapter 6: Derivative-Based. optimization 1 Chapter 6: Derivative-Based Optimization Introduction (6. Descent Methods (6. he Method of Steepest Descent (6.3 Newton s Methods (NM (6.4 Step Size Determination (6.5 Nonlinear Least-Squares Problems

More information

Nonlinear Programming

Nonlinear Programming Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week

More information

BAYESIAN ANALYSIS OF DOSE-RESPONSE CALIBRATION CURVES

BAYESIAN ANALYSIS OF DOSE-RESPONSE CALIBRATION CURVES Libraries Annual Conference on Applied Statistics in Agriculture 2005-17th Annual Conference Proceedings BAYESIAN ANALYSIS OF DOSE-RESPONSE CALIBRATION CURVES William J. Price Bahman Shafii Follow this

More information

Simulation of Gas Turbine Operations

Simulation of Gas Turbine Operations Journal of Emerging Trends in Engineering and Applied Sciences (JETEAS) (3): 49-434 Scholarlink Research Institute Journals, 011 (ISSN: 141-7016) jeteas.scholarlinkresearch.org Journal of Emerging Trends

More information

7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP.

7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques

More information

Introduction to unconstrained optimization - direct search methods

Introduction to unconstrained optimization - direct search methods Introduction to unconstrained optimization - direct search methods Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Structure of optimization methods Typically Constraint handling converts the

More information

Mathematical optimization

Mathematical optimization Optimization Mathematical optimization Determine the best solutions to certain mathematically defined problems that are under constrained determine optimality criteria determine the convergence of the

More information

Some Nonregular Designs From the Nordstrom and Robinson Code and Their Statistical Properties

Some Nonregular Designs From the Nordstrom and Robinson Code and Their Statistical Properties Some Nonregular Designs From the Nordstrom and Robinson Code and Their Statistical Properties HONGQUAN XU Department of Statistics, University of California, Los Angeles, CA 90095-1554, U.S.A. (hqxu@stat.ucla.edu)

More information

Gradient Descent. Sargur Srihari

Gradient Descent. Sargur Srihari Gradient Descent Sargur srihari@cedar.buffalo.edu 1 Topics Simple Gradient Descent/Ascent Difficulties with Simple Gradient Descent Line Search Brent s Method Conjugate Gradient Descent Weight vectors

More information

Numerical computation II. Reprojection error Bundle adjustment Family of Newtonʼs methods Statistical background Maximum likelihood estimation

Numerical computation II. Reprojection error Bundle adjustment Family of Newtonʼs methods Statistical background Maximum likelihood estimation Numerical computation II Reprojection error Bundle adjustment Family of Newtonʼs methods Statistical background Maximum likelihood estimation Reprojection error Reprojection error = Distance between the

More information

Process/product optimization using design of experiments and response surface methodology

Process/product optimization using design of experiments and response surface methodology Process/product optimization using design of experiments and response surface methodology M. Mäkelä Sveriges landbruksuniversitet Swedish University of Agricultural Sciences Department of Forest Biomaterials

More information

Two-Stage Computing Budget Allocation. Approach for Response Surface Method PENG JI

Two-Stage Computing Budget Allocation. Approach for Response Surface Method PENG JI Two-Stage Computing Budget Allocation Approach for Response Surface Method PENG JI NATIONAL UNIVERSITY OF SINGAPORE 2005 Two-Stage Computing Budget Allocation Approach for Response Surface Method PENG

More information

Parameter Estimation of Nonlinear Growth Models in Forestry

Parameter Estimation of Nonlinear Growth Models in Forestry Fekedulegn, Silva Fennica Mac 33(4) Siurtain research & Colbert notes Parameter Estimation of Nonlinear Growth s in Forestry Parameter Estimation of Nonlinear Growth s in Forestry Desta Fekedulegn, Mairitin

More information

Math 5630: Conjugate Gradient Method Hung M. Phan, UMass Lowell March 29, 2019

Math 5630: Conjugate Gradient Method Hung M. Phan, UMass Lowell March 29, 2019 Math 563: Conjugate Gradient Method Hung M. Phan, UMass Lowell March 29, 219 hroughout, A R n n is symmetric and positive definite, and b R n. 1 Steepest Descent Method We present the steepest descent

More information

Data Fitting and Uncertainty

Data Fitting and Uncertainty TiloStrutz Data Fitting and Uncertainty A practical introduction to weighted least squares and beyond With 124 figures, 23 tables and 71 test questions and examples VIEWEG+ TEUBNER IX Contents I Framework

More information

Experimental Optimization and Response Surfaces

Experimental Optimization and Response Surfaces 5 Experimental Optimization and Response Surfaces Veli-Matti Tapani Taavitsainen Helsinki Metropolia University of Applied Sciences Finland 1. Introduction Statistical design of experiments (DOE) is commonly

More information

Conjugate Gradients: Idea

Conjugate Gradients: Idea Overview Steepest Descent often takes steps in the same direction as earlier steps Wouldn t it be better every time we take a step to get it exactly right the first time? Again, in general we choose a

More information

Applied Numerical Analysis

Applied Numerical Analysis Applied Numerical Analysis Using MATLAB Second Edition Laurene V. Fausett Texas A&M University-Commerce PEARSON Prentice Hall Upper Saddle River, NJ 07458 Contents Preface xi 1 Foundations 1 1.1 Introductory

More information

Comparison learning algorithms for artificial neural network model for flood forecasting, Chiang Mai, Thailand

Comparison learning algorithms for artificial neural network model for flood forecasting, Chiang Mai, Thailand 22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 Comparison learning algorithms for artificial neural network model for

More information

Efficiency of NNBD and NNBIBD using autoregressive model

Efficiency of NNBD and NNBIBD using autoregressive model 2018; 3(3): 133-138 ISSN: 2456-1452 Maths 2018; 3(3): 133-138 2018 Stats & Maths www.mathsjournal.com Received: 17-03-2018 Accepted: 18-04-2018 S Saalini Department of Statistics, Loyola College, Chennai,

More information

January 29, Non-linear conjugate gradient method(s): Fletcher Reeves Polak Ribière January 29, 2014 Hestenes Stiefel 1 / 13

January 29, Non-linear conjugate gradient method(s): Fletcher Reeves Polak Ribière January 29, 2014 Hestenes Stiefel 1 / 13 Non-linear conjugate gradient method(s): Fletcher Reeves Polak Ribière Hestenes Stiefel January 29, 2014 Non-linear conjugate gradient method(s): Fletcher Reeves Polak Ribière January 29, 2014 Hestenes

More information

x k+1 = x k + α k p k (13.1)

x k+1 = x k + α k p k (13.1) 13 Gradient Descent Methods Lab Objective: Iterative optimization methods choose a search direction and a step size at each iteration One simple choice for the search direction is the negative gradient,

More information

Ridge analysis of mixture response surfaces

Ridge analysis of mixture response surfaces Statistics & Probability Letters 48 (2000) 3 40 Ridge analysis of mixture response surfaces Norman R. Draper a;, Friedrich Pukelsheim b a Department of Statistics, University of Wisconsin, 20 West Dayton

More information

Features and Partial Derivatives of Bertalanffy-Richards Growth Model in Forestry

Features and Partial Derivatives of Bertalanffy-Richards Growth Model in Forestry Nonlinear Analysis: Modelling and Control, 2004, Vol. 9, No. 1, 65 73 Features and Partial Derivatives of Bertalanffy-Richards Growth Model in Forestry Y.C. Lei 1, S.Y. Zhang 2 1 Research Institute of

More information

Proceedings of the. I n t e r n a t i o n a l W o r k s h o p o n I n t e r c r o p p i n g

Proceedings of the. I n t e r n a t i o n a l W o r k s h o p o n I n t e r c r o p p i n g Proceedings of the I n t e r n a t i o n a l W o r k s h o p o n I n t e r c r o p p i n g H y d e r a b a d, I n d i a 1 0-1 3 J a n u a r y 1 9 7 9 International Crops Research Institute for t h e Semi-Arid

More information

GLM models and OLS regression

GLM models and OLS regression GLM models and OLS regression Graeme Hutcheson, University of Manchester These lecture notes are based on material published in... Hutcheson, G. D. and Sofroniou, N. (1999). The Multivariate Social Scientist:

More information

DELFT UNIVERSITY OF TECHNOLOGY

DELFT UNIVERSITY OF TECHNOLOGY DELFT UNIVERSITY OF TECHNOLOGY REPORT 16-02 The Induced Dimension Reduction method applied to convection-diffusion-reaction problems R. Astudillo and M. B. van Gijzen ISSN 1389-6520 Reports of the Delft

More information

LCMRL: Improved Estimation of Quantitation Limits

LCMRL: Improved Estimation of Quantitation Limits LCMRL: Improved Estimation of Quantitation Limits John H Carson Jr., PhD Robert O Brien Steve Winslow Steve Wendelken David Munch @ Pittcon 2015 CB&I Federal Services LLC CB&I Federal Services LLC USEPA,

More information

Adaptive Beamforming Algorithms

Adaptive Beamforming Algorithms S. R. Zinka srinivasa_zinka@daiict.ac.in October 29, 2014 Outline 1 Least Mean Squares 2 Sample Matrix Inversion 3 Recursive Least Squares 4 Accelerated Gradient Approach 5 Conjugate Gradient Method Outline

More information