SPRINT Multi-Objective Model Racing
|
|
- Erika Briggs
- 5 years ago
- Views:
Transcription
1 SPRINT Multi-Objective Model Racing Tiantian Zhang, Michael Georgiopoulos, Georgios C. Anagnostopoulos Machine Learning Lab University of Central Florida
2 Overview Racing Algorithms Multi-objective Model Selection SPRINT-Race Motivation Sequential Probability Ratio Test (Non-) Dominance Inference via Dual-SPRT Experimental Results and Discussions Conclusion and Future Direction 2
3 Racing Algorithm A set of initial models Model Selection Sample validation instances o Consider an ensemble of models and o stick with the best Elimination-type Multi-Armed Bandit Evaluate all models Abandon bad performers No Stop? Yes Return the best performers 3
4 Racing Algorithm Trade-off o Benefit: Identify best model(s) Computation Time o Price to be paid: Computational cost RAs trade off model optimalit y vs. computational effort by automatically allocating comp utational resources Racing algorithms Brute force Search Process 4
5 Racing Algorithm Name Year Statistical Test Criterion Of Goodness Application Hoeffding 1994 Hoeffiding s inequality Prediction Accuracy Classification & Func. Approximation BRACE 1994 Bayesian Statistics Cross-validation Error Classification F-Race 2002 Friedman Test Function Optimization Optimal parameter of Max- Min-Ant-System for TSP Bernstein Racing 2008 Bernstein s inequality Function Optimization Policy selection in CMA-ES S-Race 2013 Sign Test Prediction Accuracy Classification 5
6 Multi-Objective Optimization Pareto Dominance (minimization case) f 2 Pareto Front f 1
7 Multi-Objective Model Selection Multi-Criteria Recommendation Story: 2 Actors: 5 Story: 3 Actors: 4 Story: 4 Actors: 3
8 Multi-Objective Model Selection Multi-task Learning
9 Multi-Objective Model Selection Motivation Model selection often involves more than one crite rion of goodness Examples Generalization performance vs. model complexity Single model addressing multiple tasks How can racing be adapted to a MOMS setting? Approaches Scalarization of criteria into a single criterion Pros: Existing racing algorithms may be applicable Cons: Good models may be left undiscovered (due to non-convexity of the relevant Pareto front) Vector Optimization Pros: Selecting models based on Pareto optimality Cons: No racing algorithm available
10 Multi-Objective Racing Algorithm S-Race Fixed-Budget Offline Maximize selection accuracy SPRINT- Race Fixed-Confidence Online Minimize computational cost
11 SPRINT-Race Racing via Sequential Probability Ratio with INdiff erence zone Test Try to minimize the computational effort needed to achieve a predefi ned confidence about the quality of the returned models. A near-optimal non-parametric ternary-decision sequenti al analogue of the sign test is adopted to identify pair-wis e dominance/non-dominance relationship. Strictly confine the error probability of returning dominate d models and, simultaneously, abandoning non-dominated models at a predefined level. Able to stop automatically. The concept of indifference zone is introduced.
12 Non-sequential VS Sequential Test Non-sequential test Sample size n is fixed Given αα and rejection region, ββ is a function of sample size. The Uniformly Most Efficient test given n and αα, find the test procedur e that minimize ββ Sequential Test Sample size n is a variable Either we accept HH 0, or we accept HH 1, or we continue sampling Properties: OC function (probability of accepting HH 0 ), ASN (average samp le number) The Uniformly Most Efficient test given αα and ββ, find the test that mini mize the expected sample size
13 SPRT -- Sequential Probability Ratio Test Locally Most Efficient Test procedure SPRT with Bernoulli Distribution At n-th step, assume xx 1, xx 2,, xx nn are i.i.d. samples collected from a Bernoulli distri bution with P xx ii = 1 = pp, and dd = # xx ii = 1 Given two simple hypothesis HH 00 : pp = pp 00 HH 11 : pp = pp 11 Let AA = 1 ββ αα ββ, BB =. 1 αα ττ nn = pp 1 dd (1 pp 1 ) nn dd pp 0 dd (1 pp 0 ) nn dd if ττ nn BB, HH 00 is accepted; if ττ nn A, HH 1 is accepted; otherwise, continue sampling.
14 (Non-) Dominance Inference Given model CC ii and CC jj. nn iijj - the number of instances that the performance vector of nn jjjj CC ii dominates the performance vector of CC jj - vise versa p the probability that CC ii dominates CC jj nn iiii ~BBBBBBBBBBBBBBBB(nn iiii + nn jjjj, pp) δδ δδ pp
15 Dual-SPRT (Sobel-Wald s test) The three-hypothesis problem is composed of two component two-hypothesis tests SPRT1: HH 0 : pp 1 δδ 2 HH 1: pp 1 2 SPRT2: HH 0 : pp 1 2 HH 1 : pp 1 + δδ 2 SPRT1 SPRT2 pp δδ pp pp pp δδ pp δδ pp = pp δδ
16 Dual-SPRT (Sobel-Wald s test) Assume that common αα and ββ values are shared between SPRT1 and SPRT2 n ij H 2 H 1 A C B H 0 n ij + n ji
17 Dual-SPRT Accuracy Analysis Interval Wrong Decisio ns γγ(pp) pp 1 2 δδ Accept HH 1 or HH 2 γγ(pp) αα 1 2 δδ < pp < 1 2 pp = 1 2 Accept HH 2 Accept HH 0 or HH 2 γγ(pp) αα γγ pp αα + ββ 1 2 < pp < δδ Accept HH 0 γγ pp < ββ pp δδ Accept HH 0 or HH 1 γγ(pp) ββ γγ = max{γγ pp } ~αα + ββ
18 Overall Error Control Bonferroni Approach ( MM 2 ) ii=1 γγ ( MM 2 ii = ) ii=1 (αα ii + ββ ii ) MM Let αα ii = ββ ii = εε for ii = 1,, 2, mmmmmm denote the maximum error probability allowed for SPRINT-Race, εε = mmmmmm MM(MM 11)
19 SPRINT-Race Initialize PPoooooo CC 1, CC 2,, CC mm mm 2, t = 1 Randomly sample a problem instance for each pair CC ii, CC jj PPPPPPPP ss. tt. ii < jj do if the corresponding dual-sprt continues then Evaluate CC ii, CC jj, update nn iiii, nn jjjj if HH 0 is accepted then PPoooooo PPPPPPPP\{CC ii } stop all dual-sprts involving CC ii else if HH 2 is accepted then PPPPPPPP PPPPPPPP\{CC jj } stop all dual-sprts involving CC jj else if HH 1 is accepted then stop all dual-sprts involving CC ii, CC jj end if end for until All dual-sprts are terminated return the Pareto front models found
20 Experiments - Artificially Constructed MOMS Problem Given M models, construct MM 2 with known p values. Three performance metrics RR rrrrrrrrrrrrrrrrrr = PP RR PP PPPP PP PPPP EE eeeeeeeeeeee = PP RR\PP PPPP PP RR PP PPFF - Pareto front models Bernoulli distributions = 1 PP RR PP PPPP PP RR PP RR - models returned by SPRINT-Race T sample complexity 20
21 Impact of No. of Objectives R E R T E 2.00E E T 21
22 Impact of δδ and mmmmmm Values 3.5 x obj 3-obj Sample Complexity (T) R+E δ δ Decreasing mmmmmm will definitely increase sample com plexity with slightly varied R and E values. 22
23 Experiments - ACO Selection for TSPs Parameter tuning of ACOs is time consuming A pool of 125 candidate models were initialized with diverse confi guration in terms on different combinations of three parameters αα AAAAAA - the influence of pheromone trials ββ AAAAAA - the influence of heuristic information ρρ AAAAAA - the pheromone trial evaporation rate Two objectives Minimize the TSP tour length Minimize the actual computation time to find this tour ACOTSPJava + DIMACS TSP instance generator 23
24 Experiments - ACO Selection for TSPs Dominated Models Non-dominated Models Normalized Computation Time Normalized Tour Length Minimum Probability of Dominance Dominated Non-dominated p= Index of Models 24
25 Conclusions SPRINT-Race is a multi-objective racing algorithm Solving multi-objective model selection Dual-SPRT is adopted for dominance and non-dominance inference The total probability of falsely retaining any dominated model and remov e any non-dominated model is strictly controlled Be able to stop automatically with fixed confidence SPRINT-Race is able to return almost exactly the true Pareto front models at a reduced cost Artificially-constructed MOMS problems Select Pareto optimal ACP parameters for solving TSPs? Optimal model configuration 25
26 References [1] O. Maron and A. Moore. Hoeffding races: Accelerating model selection search for classification and function approximation. A dvances in Neural Information Processing Systems, 6:59 66, [2] T. Zhang, M. Georgiopoulos, and G. C. Anagnostopoulos. S-Race: A multi-objective racing algorithm. In GECCO 13 Proc. of the Genet. and Evol. Comput. Conf., pages , [3] B. K. Ghosh. Sequential Tests of Statistical Hypotheses. Addison-Wesley Publishing Company, Inc., [4] Y. Jin and B. Sendhoff. Pareto-based multi-objective machine learning: An overview and case studies. IEEE Trans. Syst., Man, C ybern. C, Appl. Rev., 38: , [5] W. Mendenhall, D. D. Wackerly, and R. L. Scheaffer. Nonparametric statistics. Math. Stat. with App., pages , [6] M. Sobel and A. Wald. A sequential decision procedure for choosing one of three hypotheses concerning the unknown mean of a normal distribution. The Ann. of Math. Stat., 20: , [7] M. Birattari, Z. Yuan, P. Balaprakash, and T. Stützle. F-race and iterated F-race: An overview. Technical report, IRIDIA,
27 Thank you! Any Questions? 27
28 Backup Slides 28
29 Multi-Objective Optimization Performance vector of model C L objectives ff(cc) ff 1 (CC), ff 2 (CC),, ff LL (CC) (Minimization) CC dddddddddddddddds CC if and only if ff ii (CC) ff ii CC ii {1, 2,, LL} and jj 1, 2,, LL jj ii ff jj (CC) < ff jj CC
30 Multi-Objective Racing Algorithm S-Race (GECCO 13) Pareto-optimality Test Pareto dominance non-parametrically via sign test Consider family-wise testing procedures Limitations of S-Race Х No inference on non-dominance relationship Х No control over the overall probability of making any Ty pe II errors Х Sign test is not an optimum test procedure in sequential setting
31 Dual-SPRT (Sobel-Wald s test) The three-hypothesis problem is composed of two componen t two-hypothesis tests SPRT1: HH 0 : pp 1 2 δδ HH 1: pp 1 2 SPRT2: HH 0 : pp 1 2 HH 1 : pp δδ if SPRT1 accepts if SPRT2 accepts then dual SPRT accepts HH 0 : pp 1 2 δδ HH 0 : pp 1 2 HH 1 : pp 1 2 δδ(remove CC ii) HH 1 : pp 1 2 HH 0 : pp 1 2 HH 2 : pp = 1 2 (keep both) HH 1 : pp 1 2 HH 1 : pp εε HH 3 : pp εε (remove CC jj)
SPRINT Multi-Objective Model Racing
SPRINT Multi-Objective Model Racing Tiantian Zhang, Michael Georgiopoulos and Georgios C. Anagnostopoulos zhangtt@knights.ucf.edu, michaelg@ucf.edu and georgio@fit.edu Abstract Multi-objective model selection,
More informationModel Selection via Racing
University of Central Florida Electronic Theses and Dissertations Doctoral Dissertation (Open Access) Model Selection via Racing 2016 Tiantian Zhang University of Central Florida Find similar works at:
More informationLecture 3. STAT161/261 Introduction to Pattern Recognition and Machine Learning Spring 2018 Prof. Allie Fletcher
Lecture 3 STAT161/261 Introduction to Pattern Recognition and Machine Learning Spring 2018 Prof. Allie Fletcher Previous lectures What is machine learning? Objectives of machine learning Supervised and
More informationLecture 6. Notes on Linear Algebra. Perceptron
Lecture 6. Notes on Linear Algebra. Perceptron COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Andrey Kan Copyright: University of Melbourne This lecture Notes on linear algebra Vectors
More informationCSC 578 Neural Networks and Deep Learning
CSC 578 Neural Networks and Deep Learning Fall 2018/19 3. Improving Neural Networks (Some figures adapted from NNDL book) 1 Various Approaches to Improve Neural Networks 1. Cost functions Quadratic Cross
More informationApproximate Second Order Algorithms. Seo Taek Kong, Nithin Tangellamudi, Zhikai Guo
Approximate Second Order Algorithms Seo Taek Kong, Nithin Tangellamudi, Zhikai Guo Why Second Order Algorithms? Invariant under affine transformations e.g. stretching a function preserves the convergence
More informationVariable Selection in Predictive Regressions
Variable Selection in Predictive Regressions Alessandro Stringhi Advanced Financial Econometrics III Winter/Spring 2018 Overview This chapter considers linear models for explaining a scalar variable when
More informationEstimate by the L 2 Norm of a Parameter Poisson Intensity Discontinuous
Research Journal of Mathematics and Statistics 6: -5, 24 ISSN: 242-224, e-issn: 24-755 Maxwell Scientific Organization, 24 Submied: September 8, 23 Accepted: November 23, 23 Published: February 25, 24
More informationSimple Techniques for Improving SGD. CS6787 Lecture 2 Fall 2017
Simple Techniques for Improving SGD CS6787 Lecture 2 Fall 2017 Step Sizes and Convergence Where we left off Stochastic gradient descent x t+1 = x t rf(x t ; yĩt ) Much faster per iteration than gradient
More informationDan Roth 461C, 3401 Walnut
CIS 519/419 Applied Machine Learning www.seas.upenn.edu/~cis519 Dan Roth danroth@seas.upenn.edu http://www.cis.upenn.edu/~danroth/ 461C, 3401 Walnut Slides were created by Dan Roth (for CIS519/419 at Penn
More informationGroup-Sequential Tests for One Proportion in a Fleming Design
Chapter 126 Group-Sequential Tests for One Proportion in a Fleming Design Introduction This procedure computes power and sample size for the single-arm group-sequential (multiple-stage) designs of Fleming
More informationWorksheets for GCSE Mathematics. Algebraic Expressions. Mr Black 's Maths Resources for Teachers GCSE 1-9. Algebra
Worksheets for GCSE Mathematics Algebraic Expressions Mr Black 's Maths Resources for Teachers GCSE 1-9 Algebra Algebraic Expressions Worksheets Contents Differentiated Independent Learning Worksheets
More informationSupport Vector Machines. CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington
Support Vector Machines CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 A Linearly Separable Problem Consider the binary classification
More informationAnalytics Software. Beyond deterministic chain ladder reserves. Neil Covington Director of Solutions Management GI
Analytics Software Beyond deterministic chain ladder reserves Neil Covington Director of Solutions Management GI Objectives 2 Contents 01 Background 02 Formulaic Stochastic Reserving Methods 03 Bootstrapping
More informationAcceleration due to Gravity
1. Object Acceleration due to Gravity To determine the acceleration due to gravity by different methods. 2. Apparatus Balance, ball bearing, clamps, electric timers, meter stick, paper strips, precision
More information7.3 The Jacobi and Gauss-Seidel Iterative Methods
7.3 The Jacobi and Gauss-Seidel Iterative Methods 1 The Jacobi Method Two assumptions made on Jacobi Method: 1.The system given by aa 11 xx 1 + aa 12 xx 2 + aa 1nn xx nn = bb 1 aa 21 xx 1 + aa 22 xx 2
More informationGeneral Strong Polarization
General Strong Polarization Madhu Sudan Harvard University Joint work with Jaroslaw Blasiok (Harvard), Venkatesan Gurswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo) May 1, 018 G.Tech:
More informationMauro Birattari, Prasanna Balaprakash, and Marco Dorigo
Chapter 10 THE ACO/F-RACE ALGORITHM FOR COMBINATORIAL OPTIMIZATION UNDER UNCERTAINTY Mauro Birattari, Prasanna Balaprakash, and Marco Dorigo IRIDIA, CoDE, Université Libre de Bruxelles, Brussels, Belgium
More informationModels of collective inference
Models of collective inference Laurent Massoulié (Microsoft Research-Inria Joint Centre) Mesrob I. Ohannessian (University of California, San Diego) Alexandre Proutière (KTH Royal Institute of Technology)
More informationSolving Fuzzy Nonlinear Equations by a General Iterative Method
2062062062062060 Journal of Uncertain Systems Vol.4, No.3, pp.206-25, 200 Online at: www.jus.org.uk Solving Fuzzy Nonlinear Equations by a General Iterative Method Anjeli Garg, S.R. Singh * Department
More informationProperty Testing and Affine Invariance Part I Madhu Sudan Harvard University
Property Testing and Affine Invariance Part I Madhu Sudan Harvard University December 29-30, 2015 IITB: Property Testing & Affine Invariance 1 of 31 Goals of these talks Part I Introduce Property Testing
More information14- Hardening Soil Model with Small Strain Stiffness - PLAXIS
14- Hardening Soil Model with Small Strain Stiffness - PLAXIS This model is the Hardening Soil Model with Small Strain Stiffness as presented in PLAXIS. The model is developed using the user-defined material
More informationCSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
More informationGeneral Strong Polarization
General Strong Polarization Madhu Sudan Harvard University Joint work with Jaroslaw Blasiok (Harvard), Venkatesan Gurswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo) December 4, 2017 IAS:
More informationTuning Parameters across Mixed Dimensional Instances: A Performance Scalability Study of Sep-G-CMA-ES
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Tuning Parameters across Mixed Dimensional Instances: A Performance Scalability
More informationAngular Momentum, Electromagnetic Waves
Angular Momentum, Electromagnetic Waves Lecture33: Electromagnetic Theory Professor D. K. Ghosh, Physics Department, I.I.T., Bombay As before, we keep in view the four Maxwell s equations for all our discussions.
More informationCS Lecture 8 & 9. Lagrange Multipliers & Varitional Bounds
CS 6347 Lecture 8 & 9 Lagrange Multipliers & Varitional Bounds General Optimization subject to: min ff 0() R nn ff ii 0, h ii = 0, ii = 1,, mm ii = 1,, pp 2 General Optimization subject to: min ff 0()
More informationA Step Towards the Cognitive Radar: Target Detection under Nonstationary Clutter
A Step Towards the Cognitive Radar: Target Detection under Nonstationary Clutter Murat Akcakaya Department of Electrical and Computer Engineering University of Pittsburgh Email: akcakaya@pitt.edu Satyabrata
More informationPubh 8482: Sequential Analysis
Pubh 8482: Sequential Analysis Joseph S. Koopmeiners Division of Biostatistics University of Minnesota Week 10 Class Summary Last time... We began our discussion of adaptive clinical trials Specifically,
More informationTECHNICAL NOTE AUTOMATIC GENERATION OF POINT SPRING SUPPORTS BASED ON DEFINED SOIL PROFILES AND COLUMN-FOOTING PROPERTIES
COMPUTERS AND STRUCTURES, INC., FEBRUARY 2016 TECHNICAL NOTE AUTOMATIC GENERATION OF POINT SPRING SUPPORTS BASED ON DEFINED SOIL PROFILES AND COLUMN-FOOTING PROPERTIES Introduction This technical note
More informationCS249: ADVANCED DATA MINING
CS249: ADVANCED DATA MINING Support Vector Machine and Neural Network Instructor: Yizhou Sun yzsun@cs.ucla.edu April 24, 2017 Homework 1 Announcements Due end of the day of this Friday (11:59pm) Reminder
More informationElastic light scattering
Elastic light scattering 1. Introduction Elastic light scattering in quantum mechanics Elastic scattering is described in quantum mechanics by the Kramers Heisenberg formula for the differential cross
More informationInteraction with matter
Interaction with matter accelerated motion: ss = bb 2 tt2 tt = 2 ss bb vv = vv 0 bb tt = vv 0 2 ss bb EE = 1 2 mmvv2 dddd dddd = mm vv 0 2 ss bb 1 bb eeeeeeeeeeee llllllll bbbbbbbbbbbbbb dddddddddddddddd
More informationAdministration. CSCI567 Machine Learning (Fall 2018) Outline. Outline. HW5 is available, due on 11/18. Practice final will also be available soon.
Administration CSCI567 Machine Learning Fall 2018 Prof. Haipeng Luo U of Southern California Nov 7, 2018 HW5 is available, due on 11/18. Practice final will also be available soon. Remaining weeks: 11/14,
More informationData Mining und Maschinelles Lernen
Data Mining und Maschinelles Lernen Ensemble Methods Bias-Variance Trade-off Basic Idea of Ensembles Bagging Basic Algorithm Bagging with Costs Randomization Random Forests Boosting Stacking Error-Correcting
More informationSECTION 5: POWER FLOW. ESE 470 Energy Distribution Systems
SECTION 5: POWER FLOW ESE 470 Energy Distribution Systems 2 Introduction Nodal Analysis 3 Consider the following circuit Three voltage sources VV sss, VV sss, VV sss Generic branch impedances Could be
More informationModule 7 (Lecture 25) RETAINING WALLS
Module 7 (Lecture 25) RETAINING WALLS Topics Check for Bearing Capacity Failure Example Factor of Safety Against Overturning Factor of Safety Against Sliding Factor of Safety Against Bearing Capacity Failure
More information(1) Introduction: a new basis set
() Introduction: a new basis set In scattering, we are solving the S eq. for arbitrary VV in integral form We look for solutions to unbound states: certain boundary conditions (EE > 0, plane and spherical
More informationNon-Parametric Weighted Tests for Change in Distribution Function
American Journal of Mathematics Statistics 03, 3(3: 57-65 DOI: 0.593/j.ajms.030303.09 Non-Parametric Weighted Tests for Change in Distribution Function Abd-Elnaser S. Abd-Rabou *, Ahmed M. Gad Statistics
More informationLecture 11. Kernel Methods
Lecture 11. Kernel Methods COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Andrey Kan Copyright: University of Melbourne This lecture The kernel trick Efficient computation of a dot product
More informationNatural Computing. Lecture 11. Michael Herrmann phone: Informatics Forum /10/2011 ACO II
Natural Computing Lecture 11 Michael Herrmann mherrman@inf.ed.ac.uk phone: 0131 6 517177 Informatics Forum 1.42 25/10/2011 ACO II ACO (in brief) ACO Represent solution space Set parameters, initialize
More informationA combined approach for claiming equivalence and difference for multiple endpoints with application in GMO-trials 1
A combined approach for claiming equivalence and difference for multiple endpoints with application in GMO-trials 1 Ludwig A. Hothorn Institute of Biostatistics Leibniz University Hannover e-mail: hothorn@biostat.uni-hannover.de
More informationPerformance Evaluation and Comparison
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Cross Validation and Resampling 3 Interval Estimation
More informationKinetic Model Parameter Estimation for Product Stability: Non-uniform Finite Elements and Convexity Analysis
Kinetic Model Parameter Estimation for Product Stability: Non-uniform Finite Elements and Convexity Analysis Mark Daichendt, Lorenz Biegler Carnegie Mellon University Pittsburgh, PA 15217 Ben Weinstein,
More informationSelf-Adaptive Ant Colony System for the Traveling Salesman Problem
Proceedings of the 29 IEEE International Conference on Systems, Man, and Cybernetics San Antonio, TX, USA - October 29 Self-Adaptive Ant Colony System for the Traveling Salesman Problem Wei-jie Yu, Xiao-min
More informationEntropy Enhanced Covariance Matrix Adaptation Evolution Strategy (EE_CMAES)
1 Entropy Enhanced Covariance Matrix Adaptation Evolution Strategy (EE_CMAES) Developers: Main Author: Kartik Pandya, Dept. of Electrical Engg., CSPIT, CHARUSAT, Changa, India Co-Author: Jigar Sarda, Dept.
More informationMultiple Regression Analysis: Inference ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD
Multiple Regression Analysis: Inference ECONOMETRICS (ECON 360) BEN VAN KAMMEN, PHD Introduction When you perform statistical inference, you are primarily doing one of two things: Estimating the boundaries
More informationIntroduction to Density Estimation and Anomaly Detection. Tom Dietterich
Introduction to Density Estimation and Anomaly Detection Tom Dietterich Outline Definition and Motivations Density Estimation Parametric Density Estimation Mixture Models Kernel Density Estimation Neural
More informationA GA Mechanism for Optimizing the Design of attribute-double-sampling-plan
A GA Mechanism for Optimizing the Design of attribute-double-sampling-plan Tao-ming Cheng *, Yen-liang Chen Department of Construction Engineering, Chaoyang University of Technology, Taiwan, R.O.C. Abstract
More informationProf. Dr. Ann Nowé. Artificial Intelligence Lab ai.vub.ac.be
REINFORCEMENT LEARNING AN INTRODUCTION Prof. Dr. Ann Nowé Artificial Intelligence Lab ai.vub.ac.be REINFORCEMENT LEARNING WHAT IS IT? What is it? Learning from interaction Learning about, from, and while
More informationMultilayer Perceptrons (MLPs)
CSE 5526: Introduction to Neural Networks Multilayer Perceptrons (MLPs) 1 Motivation Multilayer networks are more powerful than singlelayer nets Example: XOR problem x 2 1 AND x o x 1 x 2 +1-1 o x x 1-1
More information(2) Orbital angular momentum
(2) Orbital angular momentum Consider SS = 0 and LL = rr pp, where pp is the canonical momentum Note: SS and LL are generators for different parts of the wave function. Note: from AA BB ii = εε iiiiii
More informationLecture 10: Logistic Regression
BIOINF 585: Machine Learning for Systems Biology & Clinical Informatics Lecture 10: Logistic Regression Jie Wang Department of Computational Medicine & Bioinformatics University of Michigan 1 Outline An
More informationSequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process
Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University
More informationCOMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16
COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS6 Lecture 3: Classification with Logistic Regression Advanced optimization techniques Underfitting & Overfitting Model selection (Training-
More informationHISTORY. In the 70 s vs today variants. (Multi-layer) Feed-forward (perceptron) Hebbian Recurrent
NEURAL NETWORKS REFERENCES: ARTIFICIAL INTELLIGENCE FOR GAMES "ARTIFICIAL INTELLIGENCE: A NEW SYNTHESIS HTTPS://MATTMAZUR.COM/2015/03/17/A- STEP- BY- STEP- BACKPROPAGATION- EXAMPLE/ " HISTORY In the 70
More informationMulti Objective Optimization
Multi Objective Optimization Handout November 4, 2011 (A good reference for this material is the book multi-objective optimization by K. Deb) 1 Multiple Objective Optimization So far we have dealt with
More informationNew Algorithms for Contextual Bandits
New Algorithms for Contextual Bandits Lev Reyzin Georgia Institute of Technology Work done at Yahoo! 1 S A. Beygelzimer, J. Langford, L. Li, L. Reyzin, R.E. Schapire Contextual Bandit Algorithms with Supervised
More informationLinear Scalarized Knowledge Gradient in the Multi-Objective Multi-Armed Bandits Problem
ESANN 04 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 3-5 April 04, i6doc.com publ., ISBN 978-8749095-7. Available from
More informationJasmin Smajic1, Christian Hafner2, Jürg Leuthold2, March 23, 2015
Jasmin Smajic, Christian Hafner 2, Jürg Leuthold 2, March 23, 205 Time Domain Finite Element Method (TD FEM): Continuous and Discontinuous Galerkin (DG-FEM) HSR - University of Applied Sciences of Eastern
More informationClassical RSA algorithm
Classical RSA algorithm We need to discuss some mathematics (number theory) first Modulo-NN arithmetic (modular arithmetic, clock arithmetic) 9 (mod 7) 4 3 5 (mod 7) congruent (I will also use = instead
More informationInternational Journal of Mathematical Archive-5(3), 2014, Available online through ISSN
International Journal of Mathematical Archive-5(3), 214, 189-195 Available online through www.ijma.info ISSN 2229 546 COMMON FIXED POINT THEOREMS FOR OCCASIONALLY WEAKLY COMPATIBLE MAPPINGS IN INTUITIONISTIC
More informationMachine Learning. Gaussian Mixture Models. Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall
Machine Learning Gaussian Mixture Models Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall 2012 1 The Generative Model POV We think of the data as being generated from some process. We assume
More informationComparison of Stochastic Soybean Yield Response Functions to Phosphorus Fertilizer
"Science Stays True Here" Journal of Mathematics and Statistical Science, Volume 016, 111 Science Signpost Publishing Comparison of Stochastic Soybean Yield Response Functions to Phosphorus Fertilizer
More informationReview for Exam Hyunse Yoon, Ph.D. Assistant Research Scientist IIHR-Hydroscience & Engineering University of Iowa
57:020 Fluids Mechanics Fall2013 1 Review for Exam3 12. 11. 2013 Hyunse Yoon, Ph.D. Assistant Research Scientist IIHR-Hydroscience & Engineering University of Iowa 57:020 Fluids Mechanics Fall2013 2 Chapter
More informationAlgebraic Codes and Invariance
Algebraic Codes and Invariance Madhu Sudan Harvard April 30, 2016 AAD3: Algebraic Codes and Invariance 1 of 29 Disclaimer Very little new work in this talk! Mainly: Ex- Coding theorist s perspective on
More informationBrief Introduction of Machine Learning Techniques for Content Analysis
1 Brief Introduction of Machine Learning Techniques for Content Analysis Wei-Ta Chu 2008/11/20 Outline 2 Overview Gaussian Mixture Model (GMM) Hidden Markov Model (HMM) Support Vector Machine (SVM) Overview
More informationAvailable online at ScienceDirect. Procedia Computer Science 20 (2013 ) 90 95
Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 20 (2013 ) 90 95 Complex Adaptive Systems, Publication 3 Cihan H. Dagli, Editor in Chief Conference Organized by Missouri
More informationLecture 4 January 23
STAT 263/363: Experimental Design Winter 2016/17 Lecture 4 January 23 Lecturer: Art B. Owen Scribe: Zachary del Rosario 4.1 Bandits Bandits are a form of online (adaptive) experiments; i.e. samples are
More informationIntroduction to Bayesian Statistics
Bayesian Parameter Estimation Introduction to Bayesian Statistics Harvey Thornburg Center for Computer Research in Music and Acoustics (CCRMA) Department of Music, Stanford University Stanford, California
More informationMathematical Methods 2019 v1.2
Problem-solving and modelling task (20%) This sample has been compiled by the QCAA to assist and support teachers to match evidence in student responses to the characteristics described in the instrument-specific
More informationLogistic Regression and Boosting for Labeled Bags of Instances
Logistic Regression and Boosting for Labeled Bags of Instances Xin Xu and Eibe Frank Department of Computer Science University of Waikato Hamilton, New Zealand {xx5, eibe}@cs.waikato.ac.nz Abstract. In
More informationPhoton Interactions in Matter
Radiation Dosimetry Attix 7 Photon Interactions in Matter Ho Kyung Kim hokyung@pusan.ac.kr Pusan National University References F. H. Attix, Introduction to Radiological Physics and Radiation Dosimetry,
More informationOn one Application of Newton s Method to Stability Problem
Journal of Multidciplinary Engineering Science Technology (JMEST) ISSN: 359-0040 Vol. Issue 5, December - 204 On one Application of Newton s Method to Stability Problem Şerife Yılmaz Department of Mathematics,
More information12. LOCAL SEARCH. gradient descent Metropolis algorithm Hopfield neural networks maximum cut Nash equilibria
12. LOCAL SEARCH gradient descent Metropolis algorithm Hopfield neural networks maximum cut Nash equilibria Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley h ttp://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationSelected Topics in Optimization. Some slides borrowed from
Selected Topics in Optimization Some slides borrowed from http://www.stat.cmu.edu/~ryantibs/convexopt/ Overview Optimization problems are almost everywhere in statistics and machine learning. Input Model
More information10.4 The Cross Product
Math 172 Chapter 10B notes Page 1 of 9 10.4 The Cross Product The cross product, or vector product, is defined in 3 dimensions only. Let aa = aa 1, aa 2, aa 3 bb = bb 1, bb 2, bb 3 then aa bb = aa 2 bb
More informationWorksheets for GCSE Mathematics. Quadratics. mr-mathematics.com Maths Resources for Teachers. Algebra
Worksheets for GCSE Mathematics Quadratics mr-mathematics.com Maths Resources for Teachers Algebra Quadratics Worksheets Contents Differentiated Independent Learning Worksheets Solving x + bx + c by factorisation
More informationLecture 3. Linear Regression
Lecture 3. Linear Regression COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Andrey Kan Copyright: University of Melbourne Weeks 2 to 8 inclusive Lecturer: Andrey Kan MS: Moscow, PhD:
More informationA linear operator of a new class of multivalent harmonic functions
International Journal of Scientific Research Publications, Volume 7, Issue 8, August 2017 278 A linear operator of a new class of multivalent harmonic functions * WaggasGalibAtshan, ** Ali Hussein Battor,
More informationThe two subset recurrent property of Markov chains
The two subset recurrent property of Markov chains Lars Holden, Norsk Regnesentral Abstract This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when
More informationDefinition: A sequence is a function from a subset of the integers (usually either the set
Math 3336 Section 2.4 Sequences and Summations Sequences Geometric Progression Arithmetic Progression Recurrence Relation Fibonacci Sequence Summations Definition: A sequence is a function from a subset
More informationReinforcement learning an introduction
Reinforcement learning an introduction Prof. Dr. Ann Nowé Computational Modeling Group AIlab ai.vub.ac.be November 2013 Reinforcement Learning What is it? Learning from interaction Learning about, from,
More informationPassing-Bablok Regression for Method Comparison
Chapter 313 Passing-Bablok Regression for Method Comparison Introduction Passing-Bablok regression for method comparison is a robust, nonparametric method for fitting a straight line to two-dimensional
More informationAMULTIOBJECTIVE optimization problem (MOP) can
1 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 Letters 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Decomposition-Based Multiobjective Evolutionary Algorithm with an Ensemble of Neighborhood Sizes Shi-Zheng
More informationRobust Multi-Objective Optimization in High Dimensional Spaces
Robust Multi-Objective Optimization in High Dimensional Spaces André Sülflow, Nicole Drechsler, and Rolf Drechsler Institute of Computer Science University of Bremen 28359 Bremen, Germany {suelflow,nd,drechsle}@informatik.uni-bremen.de
More informationCoulomb s Law and Coulomb s Constant
Pre-Lab Quiz / PHYS 224 Coulomb s Law and Coulomb s Constant Your Name: Lab Section: 1. What will you investigate in this lab? 2. Consider a capacitor created when two identical conducting plates are placed
More informationHopfield Network for Associative Memory
CSE 5526: Introduction to Neural Networks Hopfield Network for Associative Memory 1 The next few units cover unsupervised models Goal: learn the distribution of a set of observations Some observations
More informationSECTION 7: FAULT ANALYSIS. ESE 470 Energy Distribution Systems
SECTION 7: FAULT ANALYSIS ESE 470 Energy Distribution Systems 2 Introduction Power System Faults 3 Faults in three-phase power systems are short circuits Line-to-ground Line-to-line Result in the flow
More informationTests for the Odds Ratio of Two Proportions in a 2x2 Cross-Over Design
Chapter 170 Tests for the Odds Ratio of Two Proportions in a 2x2 Cross-Over Design Introduction Senn (2002) defines a cross-over design as one in which each subject receives all treatments and the objective
More informationP.3 Division of Polynomials
00 section P3 P.3 Division of Polynomials In this section we will discuss dividing polynomials. The result of division of polynomials is not always a polynomial. For example, xx + 1 divided by xx becomes
More informationThe Multi-Arm Bandit Framework
The Multi-Arm Bandit Framework A. LAZARIC (SequeL Team @INRIA-Lille) ENS Cachan - Master 2 MVA SequeL INRIA Lille MVA-RL Course In This Lecture A. LAZARIC Reinforcement Learning Algorithms Oct 29th, 2013-2/94
More informationLesson 24: True and False Number Sentences
NYS COMMON CE MATHEMATICS CURRICULUM Lesson 24 6 4 Student Outcomes Students identify values for the variables in equations and inequalities that result in true number sentences. Students identify values
More informationOn The Cauchy Problem For Some Parabolic Fractional Partial Differential Equations With Time Delays
Journal of Mathematics and System Science 6 (216) 194-199 doi: 1.17265/2159-5291/216.5.3 D DAVID PUBLISHING On The Cauchy Problem For Some Parabolic Fractional Partial Differential Equations With Time
More informationAPPROXIMATING A COMMON SOLUTION OF A FINITE FAMILY OF GENERALIZED EQUILIBRIUM AND FIXED POINT PROBLEMS
SINET: Ethiop. J. Sci., 38():7 28, 205 ISSN: 0379 2897 (Print) College of Natural Sciences, Addis Ababa University, 205 2520 7997 (Online) APPROXIMATING A COMMON SOLUTION OF A FINITE FAMILY OF GENERALIZED
More informationZebo Peng Embedded Systems Laboratory IDA, Linköping University
TDTS 01 Lecture 8 Optimization Heuristics for Synthesis Zebo Peng Embedded Systems Laboratory IDA, Linköping University Lecture 8 Optimization problems Heuristic techniques Simulated annealing Genetic
More informationReview for Exam Hyunse Yoon, Ph.D. Adjunct Assistant Professor Department of Mechanical Engineering, University of Iowa
Review for Exam2 11. 13. 2015 Hyunse Yoon, Ph.D. Adjunct Assistant Professor Department of Mechanical Engineering, University of Iowa Assistant Research Scientist IIHR-Hydroscience & Engineering, University
More informationAlgorithmisches Lernen/Machine Learning
Algorithmisches Lernen/Machine Learning Part 1: Stefan Wermter Introduction Connectionist Learning (e.g. Neural Networks) Decision-Trees, Genetic Algorithms Part 2: Norman Hendrich Support-Vector Machines
More informationA new procedure for sensitivity testing with two stress factors
A new procedure for sensitivity testing with two stress factors C.F. Jeff Wu Georgia Institute of Technology Sensitivity testing : problem formulation. Review of the 3pod (3-phase optimal design) procedure
More informationParameters affecting water hammer in metal pipelines
Parameters affecting water hammer in metal pipelines Kamil Urbanowicz 1,*, and Mateusz Firkowski 1 1 West Pomeranian University of Technology Szczecin, Department of Mechanical Engineering and Mechatronics,
More information