Bayesian Model Averaging Kriging Jize Zhang and Alexandros Taflanidis
|
|
- Lauren Jennings
- 5 years ago
- Views:
Transcription
1 HIPAD LAB: HIGH PERFORMANCE SYSTEMS LABORATORY DEPARTMENT OF CIVIL AND ENVIRONMENTAL ENGINEERING AND EARTH SCIENCES Bayesian Model Averaging Kriging Jize Zhang and Alexandros Taflanidis
2 Why use metamodeling techniques? High-fidelity numerical simulators are increasingly used in a variety of engineering alications Since they can be comutationally exensive Metamodeling techniques are adoted in many imlementations (for examle in UQ setting) to aroximate the simulator with a cheaer-to-run emulator Poular choices include: Suort vector machine Artificial neural networs Polynomial Chaos Kriging (aa Gaussian Process Regression); Data-driven statistical emulator
3 Kriging (Gaussian rocess) intro I h( x) f ( x) βz( x) Outut = Trend + GP residual T Select basis functions f and correlation ernel R Tune Kriging arameters based on exeriments (inut X/outut H air) i.e., otimize ( β,, θ ) * * * The regression structure catures deterministic global trend, through basis functions: f [ f1,, f n f ] The residual GP dictates local deviation from trend with some chosen covariance ernel: x x' cov z( x), z( x') R( ) θ
4 Kriging (Gaussian rocess) intro I h( x) f ( x) βz( x) Outut = Trend + GP residual T Select basis functions f and correlation ernel R Tune Kriging arameters based on exeriments (inut X/outut H air) h h Formulate Kriging redictions (redictive mean and variance) for any new inut x x x 1 x x 1
5 Kriging (Gaussian rocess) intro II h x x Kriging Outut rediction ~ h ( x, XH, )~ h ( x), ( x) Select basis functions f and correlation ernel R Tune Kriging arameters based on exeriments (inut X/outut H air) T T 1 Formulate Kriging redictions (redictive mean and variance) for any new inut x x 1 h ( x) f( x) β r ( x) R ( HFβ ) * * * * T T 1 1 T 1 * * * * * T 1 where uf R* r* ( x) f( x) ( x) 1 u ( F R F) u r ( x) R r ( x)
6 Is regression structure imortant? I Real model Exeriments x x 1 x x 1 Kriging with f(x) = {1} Kriging with f(x) = {x 1 x, (x ) } x x 1 x x 1 x x 1 x x 1 Kriging with f(x) = {x 3 } Kriging with f(x) = {x 4 }
7 Is regression structure imortant? II Contour lots of absolute errors (iner indicates higher error) Kriging with f(x) = {1} Kriging with f(x) = {x 1 x, x } x x x 1 x 1 x x x 1 Kriging with f(x) = {x 3 } Kriging with f(x) = {x 4 } x 1
8 Bayesian inference for regression structure selection Question: how can we select a regression structure, conditioned on all available data? Bayesian inference Terminology: Model M for Kriging with basis function vector f ={f 1 f } Bayes rule: Data PM XH H X (, ), M ( M ) Model evidence Posterior robability of each model Prior on each model
9 Bayesian model averaging rediction Reeat for every model Predict under model M h x x Kriging ( h x, XH, M ) x 1, h xxh,, M n m 1 h xxh,,, M PM ( XH, ) P( M XH, ) H X, M ( M ) Mixture weights = Posterior robability
10 Evidence evaluation Evidence: P( H X, M ) P( H X, β, σ, θ, M ) π( β, σ, θ M ) dβ dσ dθ Data lielihood Model arameter rior For each model, its evidence calculation requires a robabilistic integration over the model s arameter sace (Kriging tunable arameters), which can be numerically cumbersome (needs sohisticated MCMC samlers)
11 Evidence evaluation Evidence: P( H X, M ) P( H X, β, σ, θ, M ) π( β, σ, θ M ) dβdσ dθ Data lielihood has a determined form, because of Kriging s GP nature Model arameter rior The rior can be selected to result in analytical exression of the integral, eliminating numerical integration challenges
12 Prior elicitation I [ ] Prior selection [ β, ] 0 max(, T 1 θ, nn ) ( FF ) This rior is similar in sirit from the extremely oularused benchmar g-rior in linear regression context The red comonent is adative to both data size n and basis function size n. The blue comonent corresonds to the Fisher s information matrix of the basis function values of the training data (F ).
13 Prior elicitation II [ and θ] θ Prior selection Illustrative contour lots of conditional lielihood [ β, ] 0 max(, T 1 θ, nn ) ( FF ) [( σ ), θ ] arg ma x P( H X, M, σ, θ ) * * σ, θ Emirical Bayesian (EB) aroach: the maximizer of the conditional evidence chosen as rior; leads to closed-form exression for evidence: log P( H X, M) log P( H X, M, σ, θ) σ 1 T 1 1 T 1 HR θ H HCH log σrθ σ 1 1 n T 1 log cσ( FF ) log A log π
14 Prior elicitation II [ and θ] θ Prior selection Illustrative contour lots of conditional lielihood [ β, ] 0 max(, T 1 θ, nn ) ( FF ) [( σ ), θ ] arg ma x P( H X, M, σ, θ ) * * σ, θ Emirical Bayesian (EB) aroach: the maximizer of the conditional evidence chosen as rior; leads to closed-form exression for evidence: But Maximization is non-trivial (non-convex otimization roblem) σ
15 Retain Gaussian rediction characteristics Prior selection [ β, ] 0 max(, Not only evidence is analytically tractable log P( H X, M) log P( H X, M, σ, θ) T 1 θ, nn ) ( FF ) [( σ ), θ ] arg ma x P( H X, M, σ, θ ) * * σ, θ n log σ log c σ ( ) log log π T 1 T T 1 HR θ H HCH Rθ FF A σ But also redictions retain Gaussian nature (analytically tractable, facilitate seamless integration with existing Kriging imlementations) h( x) X, H, M, { β,, θ }~ h( x), ( x) h T BMA T 1 BMA ( x M) f( x) β r ( x) R ( HFβ ) θ * * θ 1 * ( ) ( ) T T T T x M 1 ( ) * *( ) * *( ) u c F F F R F u r x R r x θ θ θ θ
16 Comutational imlementation I Reeat for every model Predict under model M h Inefficient for large number of models (n m >>1) x x Kriging ( h x, XH, M ) x 1, h xxh,, M h, H M PM ( X, H) xx,, n m 1
17 Efficient averaging of redictions. Utilize Occam s Razor rincile, and only consider models whose lausibility is non-negligible [1]: ˆ M P ( M XH, ) α max PM ( XH, ) M M, h xxh,, h xxh,, M P( M XH, ) M ˆ where α controls the degree of restriction: α=0 : include all models; α=1 : include the most lausible one only. [1]. Raftery, A. E., Madigan, D., and Hoeting, J. A., 1997, "Bayesian model averaging for linear regression models," Journal of the American Statistical Association, 9(437),
18 Comutational imlementation II Training data X H Each model s evidence calculation requires otimization (EB arameters) intensive comutation! Model (basis functions) M 1 Model (basis functions) M... Evidence Evidence... Post rob. Post rob.... Model (basis functions) M Evidence Post rob. M P(H X, M ) P(M X, H) P( H X, M ) P M where Bayesian model class averaging * * ( H X,, σ, θ) [( σ ), θ ] arg max P( H X, M, σ, θ * * σ, θ )
19 Model sace exloration setting We assume the ool of interested basis functions are olynomial functions u to some order (set to commonly): n x nx i fc xi i i1 i1 1 1, x1,, x, 1,,, 1,, 1,, 1, n x x x n x x x x n x x n x x x x nx A oular setu because of its flexibility and hysical interretability Number of otential models is large, for examle for n x =10 dimensional roblem, the combination to exlore (for =) are 410! An efficient search imlementation is necessary
20 Efficient model search rinciles Occam s Razor: the model is ignored if its osterior robability is lesser than its simler variants (those with basis functions as its subsets) [1]. Effect Heredity: a basis function is allowed to aear only if all its arents (corresonding lower order ones) are already included []. For examle, the quadratic term (x ) is allowed only if the model already includes the corresonding constant and linear terms [1, x]. Forward search from simler to more comlex models [1]. Raftery, A. E., Madigan, D., and Hoeting, J. A., 1997, "Bayesian model averaging for linear regression models," Journal of the American Statistical Association, 9(437), []. Peixoto, J. L., 1987, "Hierarchical variable selection in olynomial regression models," The American Statistician, 41(4),
21 Forward and stewise search I Start with ordinary Kriging (OK). Calculate its EB arameters through Eq. (16) and marginal lielihood P(M 0 X,H). Set active set ={M 0 } 1st iteration ( = 1) Select model M to corresond to the th model in and obtain all allowable basis function additions (functions whose arents are included in M ) Termination Construct the final set of models by screening across ˆ M P ( H X, M ) P ( M ) α max P ( H X, M ) P ( M ) = +1 yes M no Is there a +1 model in? Weights for each PM (, ) model are XH w nˆ m M ˆ PM ( XH, ) 1 Construct the model ( M Elimination yes 1 ) with one added basis function. 1 Calculate the osterior robability PM ( XH, ) Eliminate PM XH chec PM XH 1 (, ) (, ) Reeat for each allowable basis function addition f +1 no 1 M 1 Retain M Enrich active set: 1 M Exlore if more comlex model satisfies Occam and heredity rinciles. If yes exlore around it (consider even more comlex models)
22 Intermediate elimination We can obtain (no comutational cost) aguessfor a new (comlex) M +1 model s osterior robability using the nown EB arameter of the arent M simler one: PM PM 1 1 ( X, H) ( X, H,, θ ) Comlex model EB arameters of arent simler model Intermediate elimination: ignore the comlex model without the need of calculating its EB arameter, if its aroximated osterior robability is very unromising (significantly below simler one). PM PM 1 ( XH,,, θ ) ( X, H,, θ ) Only if the model asses intermediate elimination, we identify its actual EB arameters to examine actual Occam s razor.
23 Forward and stewise search II Start with ordinary Kriging (OK). Calculate its EB arameters through Eq. (16) and marginal lielihood P(M 0 X,H). Set active set ={M 0 } 1st iteration ( = 1) Select model M to corresond to the th model in and obtain all allowable basis function additions (functions whose arents are included in M ) Termination Construct the final set of models by screening across ˆ M P ( H X, M ) P ( M ) α max P ( H X, M ) P ( M ) = +1 yes M no Is there a +1 model in? Weights for each PM (, ) model are XH w nˆ m M ˆ PM ( XH, ) 1 Construct the model ( M Intermediate elimination yes Eliminate 1 ) with one added basis function. 1 Aroximate the osterior robability PM ( XH, ) PM chec 1 ( XH, ) PM ( XH, ) Reeat for each allowable basis function addition f +1 no 1 M 1 Retain M Collect all retained models Enrich active set: no 1 M Actual Elimination chec XH 1 (, ) PM (, ) PM Reeat for each retained model M +1 XH Eliminate yes Exloration 1 Obtain the EB arameters for M through Eq. (16) 1 and the actual osterior robability PM ( XH, ) 1 M
24 Comarison methods Blind Kriging (BK): forwardly selects the next best basis function to add until the cross validation error decreases [3]. Dynamic Kriging (DK): formulate the basis function selection as an integer rogramming roblem, and sees the combination that minimizes the rocess variance with Genetic Algorithm [4]. [3]. Joseh, V. R., Hung, Y., and Sudjianto, A., 008, "Blind riging: A new method for develoing metamodels," Journal of Mechanical Design, 130(3), [4]. Zhao, L., Choi, K., and Lee, I., 011, "Metamodeling method using dynamic riging for design otimization," AIAA Journal, 49(9),
25 Comarison methods Blind Kriging (BK): forwardly selects the next best basis function to add until the cross validation error decreases [3]. Dynamic Kriging (DK): formulate the basis function selection as an integer rogramming roblem, and sees the combination that minimizes the rocess variance with Genetic Algorithm [4]. BMAK: with efficient model averaging and search strategies; Full-BMAK: roosed aroach without efficient strategies [only alicable to low-dimensional roblems]; BMSK: same as BMAK, but only use the most lausible model to redict (i.e. model selection variant)
26 Benchmar roblems I Branin: two-dimensional roblem with moderate nonlinearity Griewan: two-dimensional roblem with high nonlinearity
27 Benchmar roblems II Borehole: engineering roblem modelling the water flow rate through a borehole (Harer et al. 1983) Half-Car: realistic engineering roblem calculating the roadholding statistics of a half-car nonlinear model riding on a rough road (Zhang et al., 016). Dimension Number of exeriments Branin 14 Griewan 30 Borehole 8 4 Half-Car 19 60
28 Prediction correlation coefficient Branin Griewan Borehole Half-Car Full BMAK N/A N/A BMAK BMSK BK DK Reorted: the average rediction correlation over 50 indeendent runs Higher value Better rediction
29 More comrehensive erformance assessment Half-car examle Correlation coefficient RMS error Prediction interval coverage Better erformance Better erformance Better erformance
30 Performance for UQ redictions Borehole Half-Car BMAK BMSK BK DK Reorted: the Kolmogorov Smirnov measure between the true and rediction CDF over 50 indeendent runs Smaller discreancy Better erformance
31 Comutational cost Com. cost Branin Griewan Borehole Half-Car Tuning (EB otimization) Predictions (model averaging) Model sace size x x10 63 Conventional Kriging cost = 1 for both tuning and redictions BMAK s comutational cost is at most one magnitude greater than conventional Kriging with deterministic regression structure
32 Conclusion A Bayesian Model Averaging formulation for Kriging (BMAK) is roosed to systematically treat regression structure uncertainties. A data-driven rior is roosed, combining the benchmar g- rior and the Emirical Bayes rior; Selection lends itself to analytical tractability of roblem. Efficient model averaging and search strategy is roosed for roblems with large model sace. Results demonstrates the advantage of BMAK in both analytical and engineering roblems.
33 Than You!
AI*IA 2003 Fusion of Multiple Pattern Classifiers PART III
AI*IA 23 Fusion of Multile Pattern Classifiers PART III AI*IA 23 Tutorial on Fusion of Multile Pattern Classifiers by F. Roli 49 Methods for fusing multile classifiers Methods for fusing multile classifiers
More informationGeneral Linear Model Introduction, Classes of Linear models and Estimation
Stat 740 General Linear Model Introduction, Classes of Linear models and Estimation An aim of scientific enquiry: To describe or to discover relationshis among events (variables) in the controlled (laboratory)
More informationDistributed Rule-Based Inference in the Presence of Redundant Information
istribution Statement : roved for ublic release; distribution is unlimited. istributed Rule-ased Inference in the Presence of Redundant Information June 8, 004 William J. Farrell III Lockheed Martin dvanced
More informationEvaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models
Evaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models Ketan N. Patel, Igor L. Markov and John P. Hayes University of Michigan, Ann Arbor 48109-2122 {knatel,imarkov,jhayes}@eecs.umich.edu
More informationDETC2003/DAC AN EFFICIENT ALGORITHM FOR CONSTRUCTING OPTIMAL DESIGN OF COMPUTER EXPERIMENTS
Proceedings of DETC 03 ASME 003 Design Engineering Technical Conferences and Comuters and Information in Engineering Conference Chicago, Illinois USA, Setember -6, 003 DETC003/DAC-48760 AN EFFICIENT ALGORITHM
More informationBayesian inference & Markov chain Monte Carlo. Note 1: Many slides for this lecture were kindly provided by Paul Lewis and Mark Holder
Bayesian inference & Markov chain Monte Carlo Note 1: Many slides for this lecture were kindly rovided by Paul Lewis and Mark Holder Note 2: Paul Lewis has written nice software for demonstrating Markov
More informationA Bound on the Error of Cross Validation Using the Approximation and Estimation Rates, with Consequences for the Training-Test Split
A Bound on the Error of Cross Validation Using the Aroximation and Estimation Rates, with Consequences for the Training-Test Slit Michael Kearns AT&T Bell Laboratories Murray Hill, NJ 7974 mkearns@research.att.com
More informationBayesian Networks Practice
Bayesian Networks Practice Part 2 2016-03-17 Byoung-Hee Kim, Seong-Ho Son Biointelligence Lab, CSE, Seoul National University Agenda Probabilistic Inference in Bayesian networks Probability basics D-searation
More informationEstimating function analysis for a class of Tweedie regression models
Title Estimating function analysis for a class of Tweedie regression models Author Wagner Hugo Bonat Deartamento de Estatística - DEST, Laboratório de Estatística e Geoinformação - LEG, Universidade Federal
More informationCombining Logistic Regression with Kriging for Mapping the Risk of Occurrence of Unexploded Ordnance (UXO)
Combining Logistic Regression with Kriging for Maing the Risk of Occurrence of Unexloded Ordnance (UXO) H. Saito (), P. Goovaerts (), S. A. McKenna (2) Environmental and Water Resources Engineering, Deartment
More informationYixi Shi. Jose Blanchet. IEOR Department Columbia University New York, NY 10027, USA. IEOR Department Columbia University New York, NY 10027, USA
Proceedings of the 2011 Winter Simulation Conference S. Jain, R. R. Creasey, J. Himmelsach, K. P. White, and M. Fu, eds. EFFICIENT RARE EVENT SIMULATION FOR HEAVY-TAILED SYSTEMS VIA CROSS ENTROPY Jose
More informationUncorrelated Multilinear Principal Component Analysis for Unsupervised Multilinear Subspace Learning
TNN-2009-P-1186.R2 1 Uncorrelated Multilinear Princial Comonent Analysis for Unsuervised Multilinear Subsace Learning Haiing Lu, K. N. Plataniotis and A. N. Venetsanooulos The Edward S. Rogers Sr. Deartment
More informationarxiv: v1 [physics.data-an] 26 Oct 2012
Constraints on Yield Parameters in Extended Maximum Likelihood Fits Till Moritz Karbach a, Maximilian Schlu b a TU Dortmund, Germany, moritz.karbach@cern.ch b TU Dortmund, Germany, maximilian.schlu@cern.ch
More informationFeedback-error control
Chater 4 Feedback-error control 4.1 Introduction This chater exlains the feedback-error (FBE) control scheme originally described by Kawato [, 87, 8]. FBE is a widely used neural network based controller
More informationChapter 10. Supplemental Text Material
Chater 1. Sulemental Tet Material S1-1. The Covariance Matri of the Regression Coefficients In Section 1-3 of the tetbook, we show that the least squares estimator of β in the linear regression model y=
More informationA New GP-evolved Formulation for the Relative Permittivity of Water and Steam
ew GP-evolved Formulation for the Relative Permittivity of Water and Steam S. V. Fogelson and W. D. Potter rtificial Intelligence Center he University of Georgia, US Contact Email ddress: sergeyf1@uga.edu
More informationResearch Note REGRESSION ANALYSIS IN MARKOV CHAIN * A. Y. ALAMUTI AND M. R. MESHKANI **
Iranian Journal of Science & Technology, Transaction A, Vol 3, No A3 Printed in The Islamic Reublic of Iran, 26 Shiraz University Research Note REGRESSION ANALYSIS IN MARKOV HAIN * A Y ALAMUTI AND M R
More informationUniversal Finite Memory Coding of Binary Sequences
Deartment of Electrical Engineering Systems Universal Finite Memory Coding of Binary Sequences Thesis submitted towards the degree of Master of Science in Electrical and Electronic Engineering in Tel-Aviv
More informationRadial Basis Function Networks: Algorithms
Radial Basis Function Networks: Algorithms Introduction to Neural Networks : Lecture 13 John A. Bullinaria, 2004 1. The RBF Maing 2. The RBF Network Architecture 3. Comutational Power of RBF Networks 4.
More informationGaussian processes A hands-on tutorial
Gaussian rocesses A hands-on tutorial Slides and code: htts://github.com/araklas/gptutorial Paris Perdikaris Massachusetts Institute of Technology Deartment of Mechanical Engineering Web: htt://web.mit.edu/aris/www/
More informationUncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition
TNN-2007-P-0332.R1 1 Uncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition Haiing Lu, K.N. Plataniotis and A.N. Venetsanooulos The Edward S. Rogers
More informationUncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition
Uncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition Haiing Lu, K.N. Plataniotis and A.N. Venetsanooulos The Edward S. Rogers Sr. Deartment of
More informationEE 508 Lecture 13. Statistical Characterization of Filter Characteristics
EE 508 Lecture 3 Statistical Characterization of Filter Characteristics Comonents used to build filters are not recisely redictable L C Temerature Variations Manufacturing Variations Aging Model variations
More informationProbability Estimates for Multi-class Classification by Pairwise Coupling
Probability Estimates for Multi-class Classification by Pairwise Couling Ting-Fan Wu Chih-Jen Lin Deartment of Comuter Science National Taiwan University Taiei 06, Taiwan Ruby C. Weng Deartment of Statistics
More informationOn split sample and randomized confidence intervals for binomial proportions
On slit samle and randomized confidence intervals for binomial roortions Måns Thulin Deartment of Mathematics, Usala University arxiv:1402.6536v1 [stat.me] 26 Feb 2014 Abstract Slit samle methods have
More informationCovariance Matrix Estimation for Reinforcement Learning
Covariance Matrix Estimation for Reinforcement Learning Tomer Lancewicki Deartment of Electrical Engineering and Comuter Science University of Tennessee Knoxville, TN 37996 tlancewi@utk.edu Itamar Arel
More informationUsing the Divergence Information Criterion for the Determination of the Order of an Autoregressive Process
Using the Divergence Information Criterion for the Determination of the Order of an Autoregressive Process P. Mantalos a1, K. Mattheou b, A. Karagrigoriou b a.deartment of Statistics University of Lund
More informationAKRON: An Algorithm for Approximating Sparse Kernel Reconstruction
: An Algorithm for Aroximating Sarse Kernel Reconstruction Gregory Ditzler Det. of Electrical and Comuter Engineering The University of Arizona Tucson, AZ 8572 USA ditzler@email.arizona.edu Nidhal Carla
More informationLinear diophantine equations for discrete tomography
Journal of X-Ray Science and Technology 10 001 59 66 59 IOS Press Linear diohantine euations for discrete tomograhy Yangbo Ye a,gewang b and Jiehua Zhu a a Deartment of Mathematics, The University of Iowa,
More informationInformation collection on a graph
Information collection on a grah Ilya O. Ryzhov Warren Powell October 25, 2009 Abstract We derive a knowledge gradient olicy for an otimal learning roblem on a grah, in which we use sequential measurements
More informationInformation collection on a graph
Information collection on a grah Ilya O. Ryzhov Warren Powell February 10, 2010 Abstract We derive a knowledge gradient olicy for an otimal learning roblem on a grah, in which we use sequential measurements
More informationState Estimation with ARMarkov Models
Deartment of Mechanical and Aerosace Engineering Technical Reort No. 3046, October 1998. Princeton University, Princeton, NJ. State Estimation with ARMarkov Models Ryoung K. Lim 1 Columbia University,
More informationModeling and Estimation of Full-Chip Leakage Current Considering Within-Die Correlation
6.3 Modeling and Estimation of Full-Chi Leaage Current Considering Within-Die Correlation Khaled R. eloue, Navid Azizi, Farid N. Najm Deartment of ECE, University of Toronto,Toronto, Ontario, Canada {haled,nazizi,najm}@eecg.utoronto.ca
More informationSystem Reliability Estimation and Confidence Regions from Subsystem and Full System Tests
009 American Control Conference Hyatt Regency Riverfront, St. Louis, MO, USA June 0-, 009 FrB4. System Reliability Estimation and Confidence Regions from Subsystem and Full System Tests James C. Sall Abstract
More informationUNCERTAINLY MEASUREMENT
UNCERTAINLY MEASUREMENT Jan Čaek, Martin Ibl Institute of System Engineering and Informatics, University of Pardubice, Pardubice, Czech Reublic caek@uce.cz, martin.ibl@uce.cz In recent years, a series
More informationBayesian Spatially Varying Coefficient Models in the Presence of Collinearity
Bayesian Satially Varying Coefficient Models in the Presence of Collinearity David C. Wheeler 1, Catherine A. Calder 1 he Ohio State University 1 Abstract he belief that relationshis between exlanatory
More informationPARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH. Luoluo Liu, Trac D. Tran, and Sang Peter Chin
PARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH Luoluo Liu, Trac D. Tran, and Sang Peter Chin Det. of Electrical and Comuter Engineering, Johns Hokins Univ., Baltimore, MD 21218, USA {lliu69,trac,schin11}@jhu.edu
More informationAn Analysis of Reliable Classifiers through ROC Isometrics
An Analysis of Reliable Classifiers through ROC Isometrics Stijn Vanderlooy s.vanderlooy@cs.unimaas.nl Ida G. Srinkhuizen-Kuyer kuyer@cs.unimaas.nl Evgueni N. Smirnov smirnov@cs.unimaas.nl MICC-IKAT, Universiteit
More informationNonlinear Estimation. Professor David H. Staelin
Nonlinear Estimation Professor Davi H. Staelin Massachusetts Institute of Technology Lec22.5-1 [ DD 1 2] ˆ = 1 Best Fit, "Linear Regression" Case I: Nonlinear Physics Data Otimum Estimator P() ˆ D 1 D
More informationA Toolbox for Validation of Nonlinear Finite Element Models
A oolbox for Validation of Nonlinear Finite Element Models Dr.MarkC.Anderson ACA Inc. 279 Skyark Drive, Suite 3 orrance, CA 955-5345 Phone: (3) 53-8 Email: anderson@actainc.com imothy K. Hasselman ACA
More informationDeriving Indicator Direct and Cross Variograms from a Normal Scores Variogram Model (bigaus-full) David F. Machuca Mory and Clayton V.
Deriving ndicator Direct and Cross Variograms from a Normal Scores Variogram Model (bigaus-full) David F. Machuca Mory and Clayton V. Deutsch Centre for Comutational Geostatistics Deartment of Civil &
More informationMODEL-BASED MULTIPLE FAULT DETECTION AND ISOLATION FOR NONLINEAR SYSTEMS
MODEL-BASED MULIPLE FAUL DEECION AND ISOLAION FOR NONLINEAR SYSEMS Ivan Castillo, and homas F. Edgar he University of exas at Austin Austin, X 78712 David Hill Chemstations Houston, X 77009 Abstract A
More informationModelling 'Animal Spirits' and Network Effects in Macroeconomics and Financial Markets Thomas Lux
Modelling 'Animal Sirits' and Network Effects in Macroeconomics and Financial Markets Kiel Institute for the World Economy & Banco de Esaña Chair of Comutational Economics University of Castellón GSDP
More informationThe non-stochastic multi-armed bandit problem
Submitted for journal ublication. The non-stochastic multi-armed bandit roblem Peter Auer Institute for Theoretical Comuter Science Graz University of Technology A-8010 Graz (Austria) auer@igi.tu-graz.ac.at
More informationA MIXED CONTROL CHART ADAPTED TO THE TRUNCATED LIFE TEST BASED ON THE WEIBULL DISTRIBUTION
O P E R A T I O N S R E S E A R C H A N D D E C I S I O N S No. 27 DOI:.5277/ord73 Nasrullah KHAN Muhammad ASLAM 2 Kyung-Jun KIM 3 Chi-Hyuck JUN 4 A MIXED CONTROL CHART ADAPTED TO THE TRUNCATED LIFE TEST
More informationSTA 250: Statistics. Notes 7. Bayesian Approach to Statistics. Book chapters: 7.2
STA 25: Statistics Notes 7. Bayesian Aroach to Statistics Book chaters: 7.2 1 From calibrating a rocedure to quantifying uncertainty We saw that the central idea of classical testing is to rovide a rigorous
More informationFuzzy Automata Induction using Construction Method
Journal of Mathematics and Statistics 2 (2): 395-4, 26 ISSN 1549-3644 26 Science Publications Fuzzy Automata Induction using Construction Method 1,2 Mo Zhi Wen and 2 Wan Min 1 Center of Intelligent Control
More informationLecture 23 Maximum Likelihood Estimation and Bayesian Inference
Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Thais Paiva STA 111 - Summer 2013 Term II August 7, 2013 1 / 31 Thais Paiva STA 111 - Summer 2013 Term II Lecture 23, 08/07/2013 Lecture
More informationRobust Predictive Control of Input Constraints and Interference Suppression for Semi-Trailer System
Vol.7, No.7 (4),.37-38 htt://dx.doi.org/.457/ica.4.7.7.3 Robust Predictive Control of Inut Constraints and Interference Suression for Semi-Trailer System Zhao, Yang Electronic and Information Technology
More informationEfficient & Robust LK for Mobile Vision
Efficient & Robust LK for Mobile Vision Instructor - Simon Lucey 16-623 - Designing Comuter Vision As Direct Method (ours) Indirect Method (ORB+RANSAC) H. Alismail, B. Browning, S. Lucey Bit-Planes: Dense
More informationDIFFERENTIAL evolution (DE) [3] has become a popular
Self-adative Differential Evolution with Neighborhood Search Zhenyu Yang, Ke Tang and Xin Yao Abstract In this aer we investigate several self-adative mechanisms to imrove our revious work on [], which
More informationNotes on Instrumental Variables Methods
Notes on Instrumental Variables Methods Michele Pellizzari IGIER-Bocconi, IZA and frdb 1 The Instrumental Variable Estimator Instrumental variable estimation is the classical solution to the roblem of
More informationHotelling s Two- Sample T 2
Chater 600 Hotelling s Two- Samle T Introduction This module calculates ower for the Hotelling s two-grou, T-squared (T) test statistic. Hotelling s T is an extension of the univariate two-samle t-test
More informationESE 524 Detection and Estimation Theory
ESE 524 Detection and Estimation heory Joseh A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University 2 Urbauer
More informationA PEAK FACTOR FOR PREDICTING NON-GAUSSIAN PEAK RESULTANT RESPONSE OF WIND-EXCITED TALL BUILDINGS
The Seventh Asia-Pacific Conference on Wind Engineering, November 8-1, 009, Taiei, Taiwan A PEAK FACTOR FOR PREDICTING NON-GAUSSIAN PEAK RESULTANT RESPONSE OF WIND-EXCITED TALL BUILDINGS M.F. Huang 1,
More informationCOMPARISON OF VARIOUS OPTIMIZATION TECHNIQUES FOR DESIGN FIR DIGITAL FILTERS
NCCI 1 -National Conference on Comutational Instrumentation CSIO Chandigarh, INDIA, 19- March 1 COMPARISON OF VARIOUS OPIMIZAION ECHNIQUES FOR DESIGN FIR DIGIAL FILERS Amanjeet Panghal 1, Nitin Mittal,Devender
More informationDeveloping A Deterioration Probabilistic Model for Rail Wear
International Journal of Traffic and Transortation Engineering 2012, 1(2): 13-18 DOI: 10.5923/j.ijtte.20120102.02 Develoing A Deterioration Probabilistic Model for Rail Wear Jabbar-Ali Zakeri *, Shahrbanoo
More informationSAS for Bayesian Mediation Analysis
Paer 1569-2014 SAS for Bayesian Mediation Analysis Miočević Milica, Arizona State University; David P. MacKinnon, Arizona State University ABSTRACT Recent statistical mediation analysis research focuses
More informationFinite Mixture EFA in Mplus
Finite Mixture EFA in Mlus November 16, 2007 In this document we describe the Mixture EFA model estimated in Mlus. Four tyes of deendent variables are ossible in this model: normally distributed, ordered
More informationJournal of Chemical and Pharmaceutical Research, 2014, 6(5): Research Article
Available online www.jocr.com Journal of Chemical and harmaceutical Research, 04, 6(5):904-909 Research Article ISSN : 0975-7384 CODEN(USA) : JCRC5 Robot soccer match location rediction and the alied research
More informationEnsemble Forecasting the Number of New Car Registrations
Ensemble Forecasting the Number of New Car Registrations SJOERT FLEURKE Radiocommunications Agency Netherlands Emmasingel 1, 9700 AL, Groningen THE NETHERLANDS sjoert.fleurke@agentschatelecom.nl htt://www.agentschatelecom.nl
More informationSPARSE SIGNAL RECOVERY USING A BERNOULLI GENERALIZED GAUSSIAN PRIOR
3rd Euroean Signal Processing Conference EUSIPCO SPARSE SIGNAL RECOVERY USING A BERNOULLI GENERALIZED GAUSSIAN PRIOR Lotfi Chaari 1, Jean-Yves Tourneret 1 and Caroline Chaux 1 University of Toulouse, IRIT
More informationLower Confidence Bound for Process-Yield Index S pk with Autocorrelated Process Data
Quality Technology & Quantitative Management Vol. 1, No.,. 51-65, 15 QTQM IAQM 15 Lower onfidence Bound for Process-Yield Index with Autocorrelated Process Data Fu-Kwun Wang * and Yeneneh Tamirat Deartment
More informationA SIMPLE PLASTICITY MODEL FOR PREDICTING TRANSVERSE COMPOSITE RESPONSE AND FAILURE
THE 19 TH INTERNATIONAL CONFERENCE ON COMPOSITE MATERIALS A SIMPLE PLASTICITY MODEL FOR PREDICTING TRANSVERSE COMPOSITE RESPONSE AND FAILURE K.W. Gan*, M.R. Wisnom, S.R. Hallett, G. Allegri Advanced Comosites
More informationA Comparison between Biased and Unbiased Estimators in Ordinary Least Squares Regression
Journal of Modern Alied Statistical Methods Volume Issue Article 7 --03 A Comarison between Biased and Unbiased Estimators in Ordinary Least Squares Regression Ghadban Khalaf King Khalid University, Saudi
More informationCollaborative Place Models Supplement 1
Collaborative Place Models Sulement Ber Kaicioglu Foursquare Labs ber.aicioglu@gmail.com Robert E. Schaire Princeton University schaire@cs.rinceton.edu David S. Rosenberg P Mobile Labs david.davidr@gmail.com
More informationCSC321: 2011 Introduction to Neural Networks and Machine Learning. Lecture 10: The Bayesian way to fit models. Geoffrey Hinton
CSC31: 011 Introdution to Neural Networks and Mahine Learning Leture 10: The Bayesian way to fit models Geoffrey Hinton The Bayesian framework The Bayesian framework assumes that we always have a rior
More informationDistributed K-means over Compressed Binary Data
1 Distributed K-means over Comressed Binary Data Elsa DUPRAZ Telecom Bretagne; UMR CNRS 6285 Lab-STICC, Brest, France arxiv:1701.03403v1 [cs.it] 12 Jan 2017 Abstract We consider a networ of binary-valued
More informationConvex Optimization methods for Computing Channel Capacity
Convex Otimization methods for Comuting Channel Caacity Abhishek Sinha Laboratory for Information and Decision Systems (LIDS), MIT sinhaa@mit.edu May 15, 2014 We consider a classical comutational roblem
More informationBayesian System for Differential Cryptanalysis of DES
Available online at www.sciencedirect.com ScienceDirect IERI Procedia 7 (014 ) 15 0 013 International Conference on Alied Comuting, Comuter Science, and Comuter Engineering Bayesian System for Differential
More informationA Qualitative Event-based Approach to Multiple Fault Diagnosis in Continuous Systems using Structural Model Decomposition
A Qualitative Event-based Aroach to Multile Fault Diagnosis in Continuous Systems using Structural Model Decomosition Matthew J. Daigle a,,, Anibal Bregon b,, Xenofon Koutsoukos c, Gautam Biswas c, Belarmino
More informationDucted Wind/Water Turbines and Propellers Revisited By Michael, J. Werle, PhD 1 and Walter M. Presz, Jr., PhD 2 FLODESIGN, INC. WILBRAHAM, MA.
Introduction Ducted Wind/Water Turbines and roellers Revisited By Michael, J. Werle, hd and Walter M. resz, Jr., hd FLODEIGN, IN. WILBRAHAM, MA. 0095 There has been considerable effort and discussion in
More informationAdaptive estimation with change detection for streaming data
Adative estimation with change detection for streaming data A thesis resented for the degree of Doctor of Philosohy of the University of London and the Diloma of Imerial College by Dean Adam Bodenham Deartment
More informationAn Investigation on the Numerical Ill-conditioning of Hybrid State Estimators
An Investigation on the Numerical Ill-conditioning of Hybrid State Estimators S. K. Mallik, Student Member, IEEE, S. Chakrabarti, Senior Member, IEEE, S. N. Singh, Senior Member, IEEE Deartment of Electrical
More informationCharacterizing the Behavior of a Probabilistic CMOS Switch Through Analytical Models and Its Verification Through Simulations
Characterizing the Behavior of a Probabilistic CMOS Switch Through Analytical Models and Its Verification Through Simulations PINAR KORKMAZ, BILGE E. S. AKGUL and KRISHNA V. PALEM Georgia Institute of
More informationLecture 8 Genomic Selection
Lecture 8 Genomic Selection Guilherme J. M. Rosa University of Wisconsin-Madison Mixed Models in Quantitative Genetics SISG, Seattle 18 0 Setember 018 OUTLINE Marker Assisted Selection Genomic Selection
More informationINTRODUCTION. Please write to us at if you have any comments or ideas. We love to hear from you.
Casio FX-570ES One-Page Wonder INTRODUCTION Welcome to the world of Casio s Natural Dislay scientific calculators. Our exeriences of working with eole have us understand more about obstacles eole face
More informationAn Ant Colony Optimization Approach to the Probabilistic Traveling Salesman Problem
An Ant Colony Otimization Aroach to the Probabilistic Traveling Salesman Problem Leonora Bianchi 1, Luca Maria Gambardella 1, and Marco Dorigo 2 1 IDSIA, Strada Cantonale Galleria 2, CH-6928 Manno, Switzerland
More informationCryptanalysis of Pseudorandom Generators
CSE 206A: Lattice Algorithms and Alications Fall 2017 Crytanalysis of Pseudorandom Generators Instructor: Daniele Micciancio UCSD CSE As a motivating alication for the study of lattice in crytograhy we
More informationPairwise active appearance model and its application to echocardiography tracking
Pairwise active aearance model and its alication to echocardiograhy tracking S. Kevin Zhou 1, J. Shao 2, B. Georgescu 1, and D. Comaniciu 1 1 Integrated Data Systems, Siemens Cororate Research, Inc., Princeton,
More informationThe Poisson Regression Model
The Poisson Regression Model The Poisson regression model aims at modeling a counting variable Y, counting the number of times that a certain event occurs during a given time eriod. We observe a samle
More informationBoosting Variational Inference
Boosting Variational Inference Fangjian Guo MIT guo@csail.mit.edu Xiangyu Wang Duke University xw56@stat.duke.edu Kai Fan Duke University kai.fan@duke.edu Tamara Broderick MIT tbroderick@csail.mit.edu
More informationUnsupervised Hyperspectral Image Analysis Using Independent Component Analysis (ICA)
Unsuervised Hyersectral Image Analysis Using Indeendent Comonent Analysis (ICA) Shao-Shan Chiang Chein-I Chang Irving W. Ginsberg Remote Sensing Signal and Image Processing Laboratory Deartment of Comuter
More informationMultivariable Generalized Predictive Scheme for Gas Turbine Control in Combined Cycle Power Plant
Multivariable Generalized Predictive Scheme for Gas urbine Control in Combined Cycle Power Plant L.X.Niu and X.J.Liu Deartment of Automation North China Electric Power University Beiing, China, 006 e-mail
More informationEstimation and Detection
Estimation and Detection Lecture : Detection Theory Unknown Parameters Dr. ir. Richard C. Hendriks //05 Previous Lecture H 0 : T (x) < H : T (x) > Using detection theory, rules can be derived on how to
More informationDeformation Effect Simulation and Optimization for Double Front Axle Steering Mechanism
0 4th International Conference on Comuter Modeling and Simulation (ICCMS 0) IPCSIT vol. (0) (0) IACSIT Press, Singaore Deformation Effect Simulation and Otimization for Double Front Axle Steering Mechanism
More informationResearch Article Research on Evaluation Indicator System and Methods of Food Network Marketing Performance
Advance Journal of Food Science and Technology 7(0: 80-84 205 DOI:0.9026/afst.7.988 ISSN: 2042-4868; e-issn: 2042-4876 205 Maxwell Scientific Publication Cor. Submitted: October 7 204 Acceted: December
More informationAnalysis of Multi-Hop Emergency Message Propagation in Vehicular Ad Hoc Networks
Analysis of Multi-Ho Emergency Message Proagation in Vehicular Ad Hoc Networks ABSTRACT Vehicular Ad Hoc Networks (VANETs) are attracting the attention of researchers, industry, and governments for their
More informationSTABILITY ANALYSIS TOOL FOR TUNING UNCONSTRAINED DECENTRALIZED MODEL PREDICTIVE CONTROLLERS
STABILITY ANALYSIS TOOL FOR TUNING UNCONSTRAINED DECENTRALIZED MODEL PREDICTIVE CONTROLLERS Massimo Vaccarini Sauro Longhi M. Reza Katebi D.I.I.G.A., Università Politecnica delle Marche, Ancona, Italy
More informationSTABILITY ANALYSIS AND CONTROL OF STOCHASTIC DYNAMIC SYSTEMS USING POLYNOMIAL CHAOS. A Dissertation JAMES ROBERT FISHER
STABILITY ANALYSIS AND CONTROL OF STOCHASTIC DYNAMIC SYSTEMS USING POLYNOMIAL CHAOS A Dissertation by JAMES ROBERT FISHER Submitted to the Office of Graduate Studies of Texas A&M University in artial fulfillment
More informationA Simple Weight Decay Can Improve. Abstract. It has been observed in numerical simulations that a weight decay can improve
In Advances in Neural Information Processing Systems 4, J.E. Moody, S.J. Hanson and R.P. Limann, eds. Morgan Kaumann Publishers, San Mateo CA, 1995,. 950{957. A Simle Weight Decay Can Imrove Generalization
More informationAn Improved Calibration Method for a Chopped Pyrgeometer
96 JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY VOLUME 17 An Imroved Calibration Method for a Choed Pyrgeometer FRIEDRICH FERGG OtoLab, Ingenieurbüro, Munich, Germany PETER WENDLING Deutsches Forschungszentrum
More informationTime Domain Calculation of Vortex Induced Vibration of Long-Span Bridges by Using a Reduced-order Modeling Technique
2017 2nd International Conference on Industrial Aerodynamics (ICIA 2017) ISBN: 978-1-60595-481-3 Time Domain Calculation of Vortex Induced Vibration of Long-San Bridges by Using a Reduced-order Modeling
More informationImproved Identification of Nonlinear Dynamic Systems using Artificial Immune System
Imroved Identification of Nonlinear Dnamic Sstems using Artificial Immune Sstem Satasai Jagannath Nanda, Ganaati Panda, Senior Member IEEE and Babita Majhi Deartment of Electronics and Communication Engineering,
More informationA Closed-Form Solution to the Minimum V 2
Celestial Mechanics and Dynamical Astronomy manuscrit No. (will be inserted by the editor) Martín Avendaño Daniele Mortari A Closed-Form Solution to the Minimum V tot Lambert s Problem Received: Month
More informationEstimating Posterior Ratio for Classification: Transfer Learning from Probabilistic Perspective
Estimating Posterior Ratio for Classification: Transfer Learning from Probabilistic Persective Song Liu, Kenji Fukumizu arxiv:506.02784v3 [stat.ml] 9 Oct 205 Abstract Transfer learning assumes classifiers
More informationPaper C Exact Volume Balance Versus Exact Mass Balance in Compositional Reservoir Simulation
Paer C Exact Volume Balance Versus Exact Mass Balance in Comositional Reservoir Simulation Submitted to Comutational Geosciences, December 2005. Exact Volume Balance Versus Exact Mass Balance in Comositional
More informationMethods for Computing Marginal Data Densities from the Gibbs Output
Methods for Comuting Marginal Data Densities from the Gibbs Outut Cristina Fuentes-Albero Rutgers University Leonardo Melosi London Business School May 2012 Abstract We introduce two estimators for estimating
More informationarxiv: v3 [physics.data-an] 23 May 2011
Date: October, 8 arxiv:.7v [hysics.data-an] May -values for Model Evaluation F. Beaujean, A. Caldwell, D. Kollár, K. Kröninger Max-Planck-Institut für Physik, München, Germany CERN, Geneva, Switzerland
More informationA New Method of DDB Logical Structure Synthesis Using Distributed Tabu Search
A New Method of DDB Logical Structure Synthesis Using Distributed Tabu Search Eduard Babkin and Margarita Karunina 2, National Research University Higher School of Economics Det of nformation Systems and
More information