Modeling uncertainty in metric space. Jef Caers Stanford University Stanford, California, USA

Size: px
Start display at page:

Download "Modeling uncertainty in metric space. Jef Caers Stanford University Stanford, California, USA"

Transcription

1 Modeling uncertainty in metric space Jef Caers Stanford University Stanford, California, USA

2 Contributors Celine Scheidt (Research Associate) Kwangwon Park (PhD student)

3 Motivation Modeling uncertainty and prediction in engineering Effective? What modeling efforts matter for critical decisions? how much of present approaches are routinely used by geoscientists and engineers in the real applications? Efficient? Do our methodologies work at the scale and time framework of real projects and real time decision making?

4 State of the art Monte Carlo simulation Generating 100s of alternative models is still infeasible Mostly resort to generating a few models Important uncertainties often ignored Experimental design / response surface analysis Limited to simple variables Limited in scope of applicability

5 Modeling, distances and metric space Modeling uncertainty in metric space

6 Modeling in the Earth Sciences Data and Input variables: Geological Geophysical Engineering Model Geo processes Waves Flow Complex System Output Response prediction Optimization Control variables High Dimensional space Low Dimensional space Fit for purpose modeling Dimension reduction

7 Dimension/complexity reduction High Dimensional space Lower Dimensional space Traditional dimension reduction method (PCA) do not account for purpose One may risk discarding vital elements of the model Dimension may not be sufficiently reduced

8 Distance: metric space High Dimensional space Metric space Distance Cartesian No Axis defined Max dimension = # models Distance = some difference (scalar) between any two reservoir models What distance? Chose a distance that is correlated with the difference in response

9 Distances: multi dimensional scaling 1000 Gaussian realizations x 1 2D 2D map map of of realizations: explicit explicit visualization of of uncertainty x 2 x 1000 Calculate Euclidean distance d ij = ( x i x j ) T ( x i x j )

10 MDS Summary Reservoirmodel matrix X Dot product B = XX Reconstruction X = [ x, x, K, ] = x Euclideandistance matrix A T 1 VΛ 1/2 2 = VΛV T L

11 Application tailored distance 1000 Gaussian models 2D map of models Calculate a connectivity distance

12 Role of MDS To transform the application tailored distance into an approximating Euclidean distance (Ed) We know a lot of theory on Ed To visualize metric space by projection An explicit visual and diagnostic tool for (prior) model uncertainty, how model updating proceeds

13 Proper choice of distance leads to purpose driven structuring of models in metric space

14 Importance of purpose driven metric Euclidean distance Connectivity distance Color indicates amount of water produced

15 Important about the distance A distance or metric (difference between any two models) is NOT the same as a proxy (a measure of a single model) The better this distance is correlated to the difference in target output response, the more effective the distance approach will be (worst case=pure random)

16 Kernels: working with non Euclidean metrics Modeling uncertainty in metric space

17 Metric Space on x ϕ Feature Space on x No Axis defined ϕ 1 No Axis defined MDS MDS Kernels are a mathematical tool to map between metric spaces

18 Kernel transformations Transformation from one metric space to another does not require knowledge of ϕ, only knowledge of the dot product ϕ T ϕ ϕ T ( x) ϕ T ( y) = k( x, y) example k( x, y) = exp( ( x y) T σ ( x y) ) Approximating Euclidean distance obtained with MDS Role of the Kernel Increase dimension, seperability and linearity

19 Example 2D projection of models From metric space 2D projection of models in feature space RBF Kernel

20 Model expansion in metric space Modeling uncertainty in metric space

21 Why model expansion? Area of interest Common tasks in modeling: Model screening Model updating Conditioning models Response uncertainty Model expansion allows generating new models WITHOUT needing the original methodology/algorithms that generated the initial models

22 Karhunen Loeve expansions Gaussian realizations X Dot - product B = XX KL - expansion x new [ x, x,, x ] = K Euclidean distance matrix A T = 1 = VΛV VΛ 2 1/ 2 y T L

23 Euclidean distance based KL expansions 1000 models 3 new models KL expansion Calculate Euclidean distance

24 MDS of new models Blue dots: 1000 new models Red dots: 1000 original models both populations reflect the same uncertainty

25 Non Gaussian, non Euclidean model expansions? metric space ϕ Model expansion ϕ 1 feature space MDS MDS

26 Model expansion in feature space Model expansion feature space Φ x i a φ K ij ( x i ) X a Φ T ( x ) ( ) i x j xi x j = k( x, ) = i x j exp σ Kernel or Gram Matrix K (L L) K = V K Λ V K T K MDS Using K Φ = V K Λ 1/ 2 K Karhunen - Loeve expansion : 1 ϕ( x) = Φ b with b = VKy L ( y is a standard Gaussian vector)

27 Mapping back to metric space Model expansion? metric space ϕ ϕ 1 feature space MDS With B? MDS With K

28 Mapping back: the pre image problem MDS With B? MDS With K ϕ pre - image problem : ˆx new d = arg min ϕ( x x new d new d )-Φb new 2 2 with b new = 1 L V K y new solution : β opt i ˆx new d L opt = βi xd,i with β i= 1 i= 1 is only function of K, y new L and opt i X = 1

29 Creating a new model? L L new opt opt x = βi xi with βi = 1 i= 1 i= 1 metric space (hard data conditioning is maintained) Three methods MDS on B 1.Unconstrained method 2.Transformation method 3.Stochastic optimization method See next presentation of Celine Scheidt

30 Non-Euclidean, non-gaussian example 300 realizations generated using Boolean sampling Definition of connectivity distance 4 new realizations using non Euclidean, Non Gaussian KL expansion

31 Data conditioning in metric space Modeling uncertainty in metric space

32 A simple conditioning problem Reference permeability field Fractional flow data FWT (%) Time (days)

33 Formulating the conditioning problem in metric space The data: X = [ x x K x ] G = [ g x ) g( x ) Kg( x )] 1 2 L ( 1 2 L The problem: find x such that : data = g( x) The metric: d ij = d( g( x ), g( x i j )) i,j d i,data = d ( g( x ), data) i i The augmented data: X + = [ ] true + x x K x, x G = [ g x ) g( x ) Kg( x ), data] 1 2 L ( 1 2 L The problem in metric space find x such that : d( x x ) = d, d,true 0 with x MDS a x d

34 Illustration L= initial permeability models True earth Find the collection of models that map onto this location: model expansion in metric space

35 Note: diagnosing a wrong prior 200 initial permeability models True earth No need to even attempt to history match this data due to data model inconsistency

36 Illustration 200 initial permeability models True earth Find the collection of models that map onto this location: model expansion in metric space

37 The post image problem Model expansion metric space ϕ ϕ 1 feature space? MDS On B MDS On K?

38 The post image problem MDS On B MDS On K? find x such that : d( x x ) = d, d,true 0 with x MDS a x d y opt = arg min d( x, x ) with ϕ( x ) = Φ y new d,true new d new d 1 L V K y new Use gradual deformation to find multiple Gaussian vector solutions

39 Illustration four realizations through solving the post image problem by gradual deformation Solving the post image problem does not require any new flow simulation

40 Channelized case Reference injector Production well

41 Initial set 200 models mapped from metric space

42 Solve the post image problem 4 history matched models

43 Use of proxy distances Requires small CPU Proxy metric space Requires large CPU Actual metric space MDS MDS

44 Use of proxy distance Cluster and select models Proxy metric space Solve post image problem Actual metric space MDS MDS

45 Illustration P1 y 100 realizations of permeability (SGS) 100x100x1 y 2 wells: y O1 y 1 production well (P1),1 observation well (O1) y Objective: History match pressure at O1 y Distance: Difference in pressure at O1 for each time step y Proxy distance: Difference in pressure at O1 for last three time step 3500 Distance - Proxy 3000 ρ = Distance - Eclipse

46 Illustration MDS projection from proxy metric space Truth 7 models are selected for flow simulation

47 Illustration MDS projection from actual metric space Solve the post image problem through model expansion Construct 100 new models

48 Illustration Pressure at Time (days) 100 history matched models obtained by performing only 7 flow simulations

49 What s next? Celine: More on solving the pre image/post image problem Kwangwon: Kalman filtering in Metric space: a reservoir case study Celine: Joint construction of high resolution and coarse models in metric space Mehrdad: Multi point algorithms in metric space

50 Concluding remarks MUMS: Modeling Uncertainty in Metric Space: Powerful diagnostic tool on model uncertainty, datamodel consistency, model updating Distances allows including the purpose of modeling Working with ensembles is more effective and efficient than working on a single model at a time

Distance-based stochastic modeling: theory and applications

Distance-based stochastic modeling: theory and applications Distance-based stochastic modeling: theory and applications Jef Caers Stanford University, California, USA Energy Resources Engineering Department Abstract raditional to geostatistics is the quantification

More information

Inverting hydraulic heads in an alluvial aquifer constrained with ERT data through MPS and PPM: a case study

Inverting hydraulic heads in an alluvial aquifer constrained with ERT data through MPS and PPM: a case study Inverting hydraulic heads in an alluvial aquifer constrained with ERT data through MPS and PPM: a case study Hermans T. 1, Scheidt C. 2, Caers J. 2, Nguyen F. 1 1 University of Liege, Applied Geophysics

More information

Bootstrap confidence intervals for reservoir model selection techniques

Bootstrap confidence intervals for reservoir model selection techniques Bootstrap confidence intervals for reservoir model selection techniques Céline Scheidt and Jef Caers Department of Energy Resources Engineering Stanford University Abstract Stochastic spatial simulation

More information

Direct forecasting without full model inversion Jef Caers

Direct forecasting without full model inversion Jef Caers Direct forecasting without full model inversion Jef Caers Professor of Geological Sciences Stanford University, USA Why moving to Geological Sciences? To attract a greater diversity of students To be able

More information

Ensemble Kalman Filtering in Distance-Kernel Space

Ensemble Kalman Filtering in Distance-Kernel Space Ensemble Kalman Filtering in Distance-Kernel Space Kwangwon Park Abstract Current geostatistical simulation methods allow generating a realization that honors available data, such as hard and secondary

More information

A Stochastic Collocation based. for Data Assimilation

A Stochastic Collocation based. for Data Assimilation A Stochastic Collocation based Kalman Filter (SCKF) for Data Assimilation Lingzao Zeng and Dongxiao Zhang University of Southern California August 11, 2009 Los Angeles Outline Introduction SCKF Algorithm

More information

Maximum variance formulation

Maximum variance formulation 12.1. Principal Component Analysis 561 Figure 12.2 Principal component analysis seeks a space of lower dimensionality, known as the principal subspace and denoted by the magenta line, such that the orthogonal

More information

Reliability of Seismic Data for Hydrocarbon Reservoir Characterization

Reliability of Seismic Data for Hydrocarbon Reservoir Characterization Reliability of Seismic Data for Hydrocarbon Reservoir Characterization Geetartha Dutta (gdutta@stanford.edu) December 10, 2015 Abstract Seismic data helps in better characterization of hydrocarbon reservoirs.

More information

Updating of Uncertainty in Fractured Reservoirs driven by Geological Scenarios

Updating of Uncertainty in Fractured Reservoirs driven by Geological Scenarios Stanford Center for Reservoir Forecasting, 26th annual meeting, 8 May 2013 Updating of Uncertainty in Fractured Reservoirs driven by Geological Scenarios Andre Jung, Jef Caers, Stanford University Darryl

More information

Estimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction

Estimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction Estimation Tasks Short Course on Image Quality Matthew A. Kupinski Introduction Section 13.3 in B&M Keep in mind the similarities between estimation and classification Image-quality is a statistical concept

More information

Assessing the Value of Information from Inverse Modelling for Optimising Long-Term Oil Reservoir Performance

Assessing the Value of Information from Inverse Modelling for Optimising Long-Term Oil Reservoir Performance Assessing the Value of Information from Inverse Modelling for Optimising Long-Term Oil Reservoir Performance Eduardo Barros, TU Delft Paul Van den Hof, TU Eindhoven Jan Dirk Jansen, TU Delft 1 Oil & gas

More information

SVMs: Non-Separable Data, Convex Surrogate Loss, Multi-Class Classification, Kernels

SVMs: Non-Separable Data, Convex Surrogate Loss, Multi-Class Classification, Kernels SVMs: Non-Separable Data, Convex Surrogate Loss, Multi-Class Classification, Kernels Karl Stratos June 21, 2018 1 / 33 Tangent: Some Loose Ends in Logistic Regression Polynomial feature expansion in logistic

More information

Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen

Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen PARAMetric UNCertainties, Budapest STOCHASTIC PROCESSES AND FIELDS Noémi Friedman Institut für Wissenschaftliches Rechnen, wire@tu-bs.de

More information

Geostatistical History Matching coupled with Adaptive Stochastic Sampling: A zonation-based approach using Direct Sequential Simulation

Geostatistical History Matching coupled with Adaptive Stochastic Sampling: A zonation-based approach using Direct Sequential Simulation Geostatistical History Matching coupled with Adaptive Stochastic Sampling: A zonation-based approach using Direct Sequential Simulation Eduardo Barrela* Instituto Superior Técnico, Av. Rovisco Pais 1,

More information

COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017

COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017 COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University FEATURE EXPANSIONS FEATURE EXPANSIONS

More information

Designing Kernel Functions Using the Karhunen-Loève Expansion

Designing Kernel Functions Using the Karhunen-Loève Expansion July 7, 2004. Designing Kernel Functions Using the Karhunen-Loève Expansion 2 1 Fraunhofer FIRST, Germany Tokyo Institute of Technology, Japan 1,2 2 Masashi Sugiyama and Hidemitsu Ogawa Learning with Kernels

More information

Advances in Locally Varying Anisotropy With MDS

Advances in Locally Varying Anisotropy With MDS Paper 102, CCG Annual Report 11, 2009 ( 2009) Advances in Locally Varying Anisotropy With MDS J.B. Boisvert and C. V. Deutsch Often, geology displays non-linear features such as veins, channels or folds/faults

More information

Experiences with Model Reduction and Interpolation

Experiences with Model Reduction and Interpolation Experiences with Model Reduction and Interpolation Paul Constantine Stanford University, Sandia National Laboratories Qiqi Wang (MIT) David Gleich (Purdue) Emory University March 7, 2012 Joe Ruthruff (SNL)

More information

Contents. Acknowledgments

Contents. Acknowledgments Table of Preface Acknowledgments Notation page xii xx xxi 1 Signals and systems 1 1.1 Continuous and discrete signals 1 1.2 Unit step and nascent delta functions 4 1.3 Relationship between complex exponentials

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Kernel Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB CSE 474/574 1 / 21

More information

Distance Preservation - Part I

Distance Preservation - Part I October 2, 2007 1 Introduction 2 Scalar product Equivalence with PCA Euclidean distance 3 4 5 Spatial distances Only the coordinates of the points affects the distances. L p norm: a p = p D k=1 a k p Minkowski

More information

New Developments in LS-OPT - Robustness Studies

New Developments in LS-OPT - Robustness Studies 5. LS-DYNA - Anwenderforum, Ulm 2006 Robustheit / Optimierung I New Developments in LS-OPT - Robustness Studies Ken Craig, Willem Roux 2, Nielen Stander 2 Multidisciplinary Design Optimization Group University

More information

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University PRINCIPAL COMPONENT ANALYSIS DIMENSIONALITY

More information

Morphing ensemble Kalman filter

Morphing ensemble Kalman filter Morphing ensemble Kalman filter and applications Center for Computational Mathematics Department of Mathematical and Statistical Sciences University of Colorado Denver Supported by NSF grants CNS-0623983

More information

Kernel Methods. Barnabás Póczos

Kernel Methods. Barnabás Póczos Kernel Methods Barnabás Póczos Outline Quick Introduction Feature space Perceptron in the feature space Kernels Mercer s theorem Finite domain Arbitrary domain Kernel families Constructing new kernels

More information

Statistical Rock Physics

Statistical Rock Physics Statistical - Introduction Book review 3.1-3.3 Min Sun March. 13, 2009 Outline. What is Statistical. Why we need Statistical. How Statistical works Statistical Rock physics Information theory Statistics

More information

Data assimilation in high dimensions

Data assimilation in high dimensions Data assimilation in high dimensions David Kelly Courant Institute New York University New York NY www.dtbkelly.com February 12, 2015 Graduate seminar, CIMS David Kelly (CIMS) Data assimilation February

More information

Discriminative Direction for Kernel Classifiers

Discriminative Direction for Kernel Classifiers Discriminative Direction for Kernel Classifiers Polina Golland Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA 02139 polina@ai.mit.edu Abstract In many scientific and engineering

More information

Stochastic representation of random positive-definite tensor-valued properties: application to 3D anisotropic permeability random fields

Stochastic representation of random positive-definite tensor-valued properties: application to 3D anisotropic permeability random fields Sous la co-tutelle de : LABORATOIRE DE MODÉLISATION ET SIMULATION MULTI ÉCHELLE CNRS UPEC UNIVERSITÉ PARIS-EST CRÉTEIL UPEM UNIVERSITÉ PARIS-EST MARNE-LA-VALLÉE Stochastic representation of random positive-definite

More information

Integration of seismic and fluid-flow data: a two-way road linked by rock physics

Integration of seismic and fluid-flow data: a two-way road linked by rock physics Integration of seismic and fluid-flow data: a two-way road linked by rock physics Abstract Yunyue (Elita) Li, Yi Shen, and Peter K. Kang Geologic model building of the subsurface is a complicated and lengthy

More information

. Frobenius-Perron Operator ACC Workshop on Uncertainty Analysis & Estimation. Raktim Bhattacharya

. Frobenius-Perron Operator ACC Workshop on Uncertainty Analysis & Estimation. Raktim Bhattacharya .. Frobenius-Perron Operator 2014 ACC Workshop on Uncertainty Analysis & Estimation Raktim Bhattacharya Laboratory For Uncertainty Quantification Aerospace Engineering, Texas A&M University. uq.tamu.edu

More information

Support Vector Machines

Support Vector Machines EE 17/7AT: Optimization Models in Engineering Section 11/1 - April 014 Support Vector Machines Lecturer: Arturo Fernandez Scribe: Arturo Fernandez 1 Support Vector Machines Revisited 1.1 Strictly) Separable

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Pattern Recognition Feature Extraction Hamid R. Rabiee Jafar Muhammadi, Alireza Ghasemi, Payam Siyari Spring 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Agenda Dimensionality Reduction

More information

Employing Model Reduction for Uncertainty Visualization in the Context of CO 2 Storage Simulation

Employing Model Reduction for Uncertainty Visualization in the Context of CO 2 Storage Simulation Employing Model Reduction for Uncertainty Visualization in the Context of CO 2 Storage Simulation Marcel Hlawatsch, Sergey Oladyshkin, Daniel Weiskopf University of Stuttgart Problem setting - underground

More information

Metric-based classifiers. Nuno Vasconcelos UCSD

Metric-based classifiers. Nuno Vasconcelos UCSD Metric-based classifiers Nuno Vasconcelos UCSD Statistical learning goal: given a function f. y f and a collection of eample data-points, learn what the function f. is. this is called training. two major

More information

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials by Phillip Krahenbuhl and Vladlen Koltun Presented by Adam Stambler Multi-class image segmentation Assign a class label to each

More information

Visual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions

Visual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions Monday, June 22 Visual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions Junghyun Kwon 1, Kyoung Mu Lee 1, and Frank C. Park 2 1 Department of EECS, 2 School

More information

Preprocessing & dimensionality reduction

Preprocessing & dimensionality reduction Introduction to Data Mining Preprocessing & dimensionality reduction CPSC/AMTH 445a/545a Guy Wolf guy.wolf@yale.edu Yale University Fall 2016 CPSC 445 (Guy Wolf) Dimensionality reduction Yale - Fall 2016

More information

Introduction to Bayesian methods in inverse problems

Introduction to Bayesian methods in inverse problems Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction

More information

Explaining Results of Neural Networks by Contextual Importance and Utility

Explaining Results of Neural Networks by Contextual Importance and Utility Explaining Results of Neural Networks by Contextual Importance and Utility Kary FRÄMLING Dep. SIMADE, Ecole des Mines, 158 cours Fauriel, 42023 Saint-Etienne Cedex 2, FRANCE framling@emse.fr, tel.: +33-77.42.66.09

More information

Advanced Machine Learning & Perception

Advanced Machine Learning & Perception Advanced Machine Learning & Perception Instructor: Tony Jebara Topic 6 Standard Kernels Unusual Input Spaces for Kernels String Kernels Probabilistic Kernels Fisher Kernels Probability Product Kernels

More information

Practical Bayesian Optimization of Machine Learning. Learning Algorithms

Practical Bayesian Optimization of Machine Learning. Learning Algorithms Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that

More information

4 Bias-Variance for Ridge Regression (24 points)

4 Bias-Variance for Ridge Regression (24 points) Implement Ridge Regression with λ = 0.00001. Plot the Squared Euclidean test error for the following values of k (the dimensions you reduce to): k = {0, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500,

More information

A Spectral Approach to Linear Bayesian Updating

A Spectral Approach to Linear Bayesian Updating A Spectral Approach to Linear Bayesian Updating Oliver Pajonk 1,2, Bojana V. Rosic 1, Alexander Litvinenko 1, and Hermann G. Matthies 1 1 Institute of Scientific Computing, TU Braunschweig, Germany 2 SPT

More information

Oil Field Production using Machine Learning. CS 229 Project Report

Oil Field Production using Machine Learning. CS 229 Project Report Oil Field Production using Machine Learning CS 229 Project Report Sumeet Trehan, Energy Resources Engineering, Stanford University 1 Introduction Effective management of reservoirs motivates oil and gas

More information

Global Sensitivity Analysis of Complex Systems implications for natural resources. Jef Caers Geological Sciences Stanford University, USA

Global Sensitivity Analysis of Complex Systems implications for natural resources. Jef Caers Geological Sciences Stanford University, USA Global Sensitivity Analysis of Complex Systems implications for natural resources Jef Caers Geological Sciences Stanford University, USA Analysis of (Subsurface) Systems The model: description of the world

More information

Cross-validation methods for quality control, cloud screening, etc.

Cross-validation methods for quality control, cloud screening, etc. Cross-validation methods for quality control, cloud screening, etc. Olaf Stiller, Deutscher Wetterdienst Are observations consistent Sensitivity functions with the other observations? given the background

More information

Best Practice Reservoir Characterization for the Alberta Oil Sands

Best Practice Reservoir Characterization for the Alberta Oil Sands Best Practice Reservoir Characterization for the Alberta Oil Sands Jason A. McLennan and Clayton V. Deutsch Centre for Computational Geostatistics (CCG) Department of Civil and Environmental Engineering

More information

Computational and Statistical Learning Theory

Computational and Statistical Learning Theory Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 17: Stochastic Optimization Part II: Realizable vs Agnostic Rates Part III: Nearest Neighbor Classification Stochastic

More information

PRECONDITIONING MARKOV CHAIN MONTE CARLO SIMULATIONS USING COARSE-SCALE MODELS

PRECONDITIONING MARKOV CHAIN MONTE CARLO SIMULATIONS USING COARSE-SCALE MODELS PRECONDITIONING MARKOV CHAIN MONTE CARLO SIMULATIONS USING COARSE-SCALE MODELS Y. EFENDIEV, T. HOU, AND W. LUO Abstract. We study the preconditioning of Markov Chain Monte Carlo (MCMC) methods using coarse-scale

More information

Data Assimilation Research Testbed Tutorial

Data Assimilation Research Testbed Tutorial Data Assimilation Research Testbed Tutorial Section 2: How should observations of a state variable impact an unobserved state variable? Multivariate assimilation. Single observed variable, single unobserved

More information

Principal Components Analysis. Sargur Srihari University at Buffalo

Principal Components Analysis. Sargur Srihari University at Buffalo Principal Components Analysis Sargur Srihari University at Buffalo 1 Topics Projection Pursuit Methods Principal Components Examples of using PCA Graphical use of PCA Multidimensional Scaling Srihari 2

More information

PCA, Kernel PCA, ICA

PCA, Kernel PCA, ICA PCA, Kernel PCA, ICA Learning Representations. Dimensionality Reduction. Maria-Florina Balcan 04/08/2015 Big & High-Dimensional Data High-Dimensions = Lot of Features Document classification Features per

More information

Dynamic System Identification using HDMR-Bayesian Technique

Dynamic System Identification using HDMR-Bayesian Technique Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in

More information

LECTURE NOTE #11 PROF. ALAN YUILLE

LECTURE NOTE #11 PROF. ALAN YUILLE LECTURE NOTE #11 PROF. ALAN YUILLE 1. NonLinear Dimension Reduction Spectral Methods. The basic idea is to assume that the data lies on a manifold/surface in D-dimensional space, see figure (1) Perform

More information

Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.

Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations. Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,

More information

Course on Inverse Problems

Course on Inverse Problems Stanford University School of Earth Sciences Course on Inverse Problems Albert Tarantola Third Lesson: Probability (Elementary Notions) Let u and v be two Cartesian parameters (then, volumetric probabilities

More information

Kernel-Based Contrast Functions for Sufficient Dimension Reduction

Kernel-Based Contrast Functions for Sufficient Dimension Reduction Kernel-Based Contrast Functions for Sufficient Dimension Reduction Michael I. Jordan Departments of Statistics and EECS University of California, Berkeley Joint work with Kenji Fukumizu and Francis Bach

More information

Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Linear classifier Which classifier? x 2 x 1 2 Linear classifier Margin concept x 2

More information

Uncertainty Quantification and Validation Using RAVEN. A. Alfonsi, C. Rabiti. Risk-Informed Safety Margin Characterization. https://lwrs.inl.

Uncertainty Quantification and Validation Using RAVEN. A. Alfonsi, C. Rabiti. Risk-Informed Safety Margin Characterization. https://lwrs.inl. Risk-Informed Safety Margin Characterization Uncertainty Quantification and Validation Using RAVEN https://lwrs.inl.gov A. Alfonsi, C. Rabiti North Carolina State University, Raleigh 06/28/2017 Assumptions

More information

RESERVOIR MODELING & CHARACTERIZATION TOOLS: APPLYING METHODS & TOOLS FROM THE OIL AND GAS INDUSTRY TO ENHANCE GEOTHERMAL RESOURCES.

RESERVOIR MODELING & CHARACTERIZATION TOOLS: APPLYING METHODS & TOOLS FROM THE OIL AND GAS INDUSTRY TO ENHANCE GEOTHERMAL RESOURCES. RESERVOIR MODELING & CHARACTERIZATION TOOLS: APPLYING METHODS & TOOLS FROM THE OIL AND GAS INDUSTRY TO ENHANCE GEOTHERMAL RESOURCES Jacques CHOURAKI 1 NEW TOOLS & METHODS CURRENTLY, GEOSCIENTIFIC STUDIES,

More information

The Gram Schmidt Process

The Gram Schmidt Process u 2 u The Gram Schmidt Process Now we will present a procedure, based on orthogonal projection, that converts any linearly independent set of vectors into an orthogonal set. Let us begin with the simple

More information

The Gram Schmidt Process

The Gram Schmidt Process The Gram Schmidt Process Now we will present a procedure, based on orthogonal projection, that converts any linearly independent set of vectors into an orthogonal set. Let us begin with the simple case

More information

Effect of velocity uncertainty on amplitude information

Effect of velocity uncertainty on amplitude information Stanford Exploration Project, Report 111, June 9, 2002, pages 253 267 Short Note Effect of velocity uncertainty on amplitude information Robert G. Clapp 1 INTRODUCTION Risk assessment is a key component

More information

A new Hierarchical Bayes approach to ensemble-variational data assimilation

A new Hierarchical Bayes approach to ensemble-variational data assimilation A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander

More information

Gaussian Process Approximations of Stochastic Differential Equations

Gaussian Process Approximations of Stochastic Differential Equations Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run

More information

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 59 CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 4. INTRODUCTION Weighted average-based fusion algorithms are one of the widely used fusion methods for multi-sensor data integration. These methods

More information

Beyond the Point Cloud: From Transductive to Semi-Supervised Learning

Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Vikas Sindhwani, Partha Niyogi, Mikhail Belkin Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of

More information

Implicit sampling for particle filters. Alexandre Chorin, Mathias Morzfeld, Xuemin Tu, Ethan Atkins

Implicit sampling for particle filters. Alexandre Chorin, Mathias Morzfeld, Xuemin Tu, Ethan Atkins 0/20 Implicit sampling for particle filters Alexandre Chorin, Mathias Morzfeld, Xuemin Tu, Ethan Atkins University of California at Berkeley 2/20 Example: Try to find people in a boat in the middle of

More information

Robotics 2 Target Tracking. Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard

Robotics 2 Target Tracking. Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard Robotics 2 Target Tracking Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard Slides by Kai Arras, Gian Diego Tipaldi, v.1.1, Jan 2012 Chapter Contents Target Tracking Overview Applications

More information

Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA

Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA Yoshua Bengio Pascal Vincent Jean-François Paiement University of Montreal April 2, Snowbird Learning 2003 Learning Modal Structures

More information

Ensemble Kalman filter assimilation of transient groundwater flow data via stochastic moment equations

Ensemble Kalman filter assimilation of transient groundwater flow data via stochastic moment equations Ensemble Kalman filter assimilation of transient groundwater flow data via stochastic moment equations Alberto Guadagnini (1,), Marco Panzeri (1), Monica Riva (1,), Shlomo P. Neuman () (1) Department of

More information

Comparing the gradual deformation with the probability perturbation method

Comparing the gradual deformation with the probability perturbation method Comparing the gradual deformation with the probability perturbation method Jef Caers Petroleum Engineering Department Stanford University, California USA 1. Introduction This paper is intended as a comparison

More information

Gaussian Process Approximations of Stochastic Differential Equations

Gaussian Process Approximations of Stochastic Differential Equations Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Centre for Computational Statistics and Machine Learning University College London c.archambeau@cs.ucl.ac.uk CSML

More information

Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation)

Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) PCA transforms the original input space into a lower dimensional space, by constructing dimensions that are linear combinations

More information

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Philipp Krähenbühl and Vladlen Koltun Stanford University Presenter: Yuan-Ting Hu 1 Conditional Random Field (CRF) E x I = φ u

More information

Neural networks and optimization

Neural networks and optimization Neural networks and optimization Nicolas Le Roux Criteo 18/05/15 Nicolas Le Roux (Criteo) Neural networks and optimization 18/05/15 1 / 85 1 Introduction 2 Deep networks 3 Optimization 4 Convolutional

More information

Advanced Machine Learning & Perception

Advanced Machine Learning & Perception Advanced Machine Learning & Perception Instructor: Tony Jebara Topic 1 Introduction, researchy course, latest papers Going beyond simple machine learning Perception, strange spaces, images, time, behavior

More information

Kalman Filter Computer Vision (Kris Kitani) Carnegie Mellon University

Kalman Filter Computer Vision (Kris Kitani) Carnegie Mellon University Kalman Filter 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University Examples up to now have been discrete (binary) random variables Kalman filtering can be seen as a special case of a temporal

More information

The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space

The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space Robert Jenssen, Deniz Erdogmus 2, Jose Principe 2, Torbjørn Eltoft Department of Physics, University of Tromsø, Norway

More information

Expectation Propagation for Approximate Bayesian Inference

Expectation Propagation for Approximate Bayesian Inference Expectation Propagation for Approximate Bayesian Inference José Miguel Hernández Lobato Universidad Autónoma de Madrid, Computer Science Department February 5, 2007 1/ 24 Bayesian Inference Inference Given

More information

THE VINE COPULA METHOD FOR REPRESENTING HIGH DIMENSIONAL DEPENDENT DISTRIBUTIONS: APPLICATION TO CONTINUOUS BELIEF NETS

THE VINE COPULA METHOD FOR REPRESENTING HIGH DIMENSIONAL DEPENDENT DISTRIBUTIONS: APPLICATION TO CONTINUOUS BELIEF NETS Proceedings of the 00 Winter Simulation Conference E. Yücesan, C.-H. Chen, J. L. Snowdon, and J. M. Charnes, eds. THE VINE COPULA METHOD FOR REPRESENTING HIGH DIMENSIONAL DEPENDENT DISTRIBUTIONS: APPLICATION

More information

Least Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD

Least Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD Least Squares Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 75A Winter 0 - UCSD (Unweighted) Least Squares Assume linearity in the unnown, deterministic model parameters Scalar, additive noise model: y f (

More information

Apprentissage non supervisée

Apprentissage non supervisée Apprentissage non supervisée Cours 3 Higher dimensions Jairo Cugliari Master ECD 2015-2016 From low to high dimension Density estimation Histograms and KDE Calibration can be done automacally But! Let

More information

Optimisation under Uncertainty with Stochastic PDEs for the History Matching Problem in Reservoir Engineering

Optimisation under Uncertainty with Stochastic PDEs for the History Matching Problem in Reservoir Engineering Optimisation under Uncertainty with Stochastic PDEs for the History Matching Problem in Reservoir Engineering Hermann G. Matthies Technische Universität Braunschweig wire@tu-bs.de http://www.wire.tu-bs.de

More information

USING GEOSTATISTICS TO DESCRIBE COMPLEX A PRIORI INFORMATION FOR INVERSE PROBLEMS THOMAS M. HANSEN 1,2, KLAUS MOSEGAARD 2 and KNUD S.

USING GEOSTATISTICS TO DESCRIBE COMPLEX A PRIORI INFORMATION FOR INVERSE PROBLEMS THOMAS M. HANSEN 1,2, KLAUS MOSEGAARD 2 and KNUD S. USING GEOSTATISTICS TO DESCRIBE COMPLEX A PRIORI INFORMATION FOR INVERSE PROBLEMS THOMAS M. HANSEN 1,2, KLAUS MOSEGAARD 2 and KNUD S. CORDUA 1 1 Institute of Geography & Geology, University of Copenhagen,

More information

Least Squares Estimation Namrata Vaswani,

Least Squares Estimation Namrata Vaswani, Least Squares Estimation Namrata Vaswani, namrata@iastate.edu Least Squares Estimation 1 Recall: Geometric Intuition for Least Squares Minimize J(x) = y Hx 2 Solution satisfies: H T H ˆx = H T y, i.e.

More information

OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES

OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogustaw Cyganek AGH University of Science and Technology, Poland WILEY A John Wiley &. Sons, Ltd., Publication Contents Preface Acknowledgements

More information

Analysis of the Pattern Correlation between Time Lapse Seismic Amplitudes and Saturation

Analysis of the Pattern Correlation between Time Lapse Seismic Amplitudes and Saturation Analysis of the Pattern Correlation between Time Lapse Seismic Amplitudes and Saturation Darkhan Kuralkhanov and Tapan Mukerji Department of Energy Resources Engineering Stanford University Abstract The

More information

Reducing Uncertainty in Modelling Fluvial Reservoirs by using Intelligent Geological Priors

Reducing Uncertainty in Modelling Fluvial Reservoirs by using Intelligent Geological Priors Reducing Uncertainty in Modelling Fluvial Reservoirs by using Intelligent Geological Priors Temístocles Rojas 1, Vasily Demyanov 2, Mike Christie 3 & Dan Arnold 4 Abstract Automatic history matching reservoir

More information

Abstract. 1 Introduction. Cointerpretation of Flow Rate-Pressure-Temperature Data from Permanent Downhole Gauges. Deconvolution. Breakpoint detection

Abstract. 1 Introduction. Cointerpretation of Flow Rate-Pressure-Temperature Data from Permanent Downhole Gauges. Deconvolution. Breakpoint detection Cointerpretation of Flow Rate-Pressure-Temperature Data from Permanent Downhole Gauges CS 229 Course Final Report Chuan Tian chuant@stanford.edu Yue Li yuel@stanford.edu Abstract This report documents

More information

Dimension Reduction and Low-dimensional Embedding

Dimension Reduction and Low-dimensional Embedding Dimension Reduction and Low-dimensional Embedding Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208 http://www.eecs.northwestern.edu/~yingwu 1/26 Dimension

More information

Sequential Monte Carlo Samplers for Applications in High Dimensions

Sequential Monte Carlo Samplers for Applications in High Dimensions Sequential Monte Carlo Samplers for Applications in High Dimensions Alexandros Beskos National University of Singapore KAUST, 26th February 2014 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Alex

More information

Intelligent Systems:

Intelligent Systems: Intelligent Systems: Undirected Graphical models (Factor Graphs) (2 lectures) Carsten Rother 15/01/2015 Intelligent Systems: Probabilistic Inference in DGM and UGM Roadmap for next two lectures Definition

More information

Dinesh Kumar, Mehrdad Raisee and Chris Lacor

Dinesh Kumar, Mehrdad Raisee and Chris Lacor Dinesh Kumar, Mehrdad Raisee and Chris Lacor Fluid Mechanics and Thermodynamics Research Group Vrije Universiteit Brussel, BELGIUM dkumar@vub.ac.be; m_raisee@yahoo.com; chris.lacor@vub.ac.be October, 2014

More information

Branch-and-Bound Algorithm. Pattern Recognition XI. Michal Haindl. Outline

Branch-and-Bound Algorithm. Pattern Recognition XI. Michal Haindl. Outline Branch-and-Bound Algorithm assumption - can be used if a feature selection criterion satisfies the monotonicity property monotonicity property - for nested feature sets X j related X 1 X 2... X l the criterion

More information

Bayesian Dynamic Linear Modelling for. Complex Computer Models

Bayesian Dynamic Linear Modelling for. Complex Computer Models Bayesian Dynamic Linear Modelling for Complex Computer Models Fei Liu, Liang Zhang, Mike West Abstract Computer models may have functional outputs. With no loss of generality, we assume that a single computer

More information

Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems

Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Krzysztof Fidkowski, David Galbally*, Karen Willcox* (*MIT) Computational Aerospace Sciences Seminar Aerospace Engineering

More information

Towards a probabilistic hydrological forecasting and data assimilation system. Henrik Madsen DHI, Denmark

Towards a probabilistic hydrological forecasting and data assimilation system. Henrik Madsen DHI, Denmark Towards a probabilistic hydrological forecasting and data assimilation system Henrik Madsen DHI, Denmark Outline Hydrological forecasting Data assimilation framework Data assimilation experiments Concluding

More information

Introduction to Machine Learning

Introduction to Machine Learning 1, DATA11002 Introduction to Machine Learning Lecturer: Teemu Roos TAs: Ville Hyvönen and Janne Leppä-aho Department of Computer Science University of Helsinki (based in part on material by Patrik Hoyer

More information