IMPLICIT CROWDS: OPTIMIZATION INTEGRATOR FOR ROBUST CROWD SIMULATION

Size: px
Start display at page:

Download "IMPLICIT CROWDS: OPTIMIZATION INTEGRATOR FOR ROBUST CROWD SIMULATION"

Transcription

1 IMPLICIT CROWDS: OPTIMIZATION INTEGRATOR FOR ROBUST CROWD SIMULATION Ioannis Karamouzas 1, Nick Sohre 2, Rahul Narain 2, Stephen J. Guy 2 1 Clemson University 2 University of Minnesota

2 COLLISION AVOIDANCE IN CROWDS Given desired velocities, how should agents navigate around each other? 2

3 COLLISION AVOIDANCE IN CROWDS Given desired velocities, how should agents navigate around each other? 3

4 LOCAL COLLISION AVOIDANCE Adding Realism Vision-based approaches [Ondřej et al. 2010, Kapadia et al. 2012, Hughes et al. 2015, Dutra et al. 2017] Probabilistic approaches [Wolinski et al. 2016].. [Yu et al. 2012] [Wolinski et al. 2016] [Kapadia et al. 2012] 4

5 LOCAL COLLISION AVOIDANCE Force-based methods [Reynolds 1987, 1999; Helbing et al. 2000; Pelechano et al. 2007, ] Require very small time steps for stability Velocity-based methods [van den Berg et al. 2008, 2011; Pettré et al 2011, ] Overly conservative behavior [8agent/ttc_0.25] Δt = 0.1s ORCA 5

6 DESIDERATA We seek a generic technique for multi-agent navigation that guarantees collision-free motion is robust to variations in scenario, density, time step exhibits high-fidelity behavior can update at footstep rates ( s) 6

7 OUR CONTRIBUTIONS 1. General form of collision avoidance behaviors dv R x, v = dt v supporting optimization-based implicit integration 2. Application to state-of-the-art power law model R x, v τ(x, v) p for practical crowd simulations 7

8 I. OPTIMIZATION INTEGRATOR FOR CROWDS 8

9 IMPLICIT INTEGRATION d dt x v = v M 1 f(x, v) x n+1 x n = v n+1 t, M v n+1 v n = f n+1 t Unconditionally stable, but [Baraff and Witkin, 1998] 9

10 IMPLICIT INTEGRATION d dt x v = v M 1 f(x, v) x n+1 x n = v n+1 t, M v n+1 v n = f n+1 t Unconditionally stable, but Need to solve a non linear system Slow (but we can use large time steps) [Kaufman et al. 2014] 10

11 OPTIMIZATION INTEGRATORS As long as forces are conservative: f x = du x dx, we can express backward Euler in optimization form [Martin et al. 2011; Gast et al. 2015]: [Bouaziz et al. 2014] x n+1 = arg min x 1 2 t 2 x x M 2 + U(x) Interpretation: tradeoff between conserving momentum and reducing potential energy [Martin et al. 2011] 11

12 OPTIMIZATION INTEGRATORS As long as forces are conservative: f x = du x dx, we can express backward Euler in optimization form [Martin et al. 2011; Gast et al. 2015]: [Bouaziz et al. 2014] x n+1 = arg min x 1 2 t 2 x x M 2 + U(x) Interpretation: tradeoff between maintaining velocity and reducing potential energy [Martin et al. 2011] 12

13 OPTIMIZATION INTEGRATORS Why this is good: x n+1 = arg min x 1 2 t 2 x x M 2 + U(x) Simple and fast algorithms (e.g. gradient descent, Gauss- Seidel) can be given guarantees Highly nonlinear forces can be used without linearization Lots of recent advances in optimization for data mining, machine learning, image processing, [Fratarcangeli et al. 2016] 13

14 NON-CONSERVATIVE FORCES Conservative potentials U(x) can only model position-dependent forces 14

15 NON-CONSERVATIVE FORCES Conservative potentials U(x) can only model position-dependent forces Humans anticipate People anticipate future trajectories of others [Cutting et al. 2005, Olivier et al. 2012; Karamouzas et al. 2014] Brains have special neurons for estimating collisions [Gabbiani 2002] Crowd forces depend both on positions and velocities! 15

16 ANTICIPATORY FORCES Hypothesis: Multi-agent interactions can be expressed as R x, v f x, v = v where R is an anticipatory potential that drives agents away from high-cost velocities (This is analogous to dissipation potentials [Goldstein 1980] in classical mechanics) 16

17 OPTIMIZATION INTEGRATOR FOR NON-CONSERVATIVE FORCES Anticipatory forces v n+1 = arg min v 1 2 v v M 2 + R(x + v t, v) t First-order accurate, equivalent to previous formula if R = 0 U( ) + R( ) t is analogous to the effective interaction potential in symplectic integrators [Kane et al. 2000; Kharevych et al. 2006] Simple interpretation: tradeoff between conserving momentum, reducing U, and reducing R 17

18 OPTIMIZATION INTEGRATOR FOR NON-CONSERVATIVE FORCES Anticipatory + conservative forces v n+1 = arg min v First-order accurate 1 2 v v M 2 + U(x + v t) + R(x + v t, v) t U( ) + R( ) t is analogous to the effective interaction potential in symplectic integrators [Kane et al. 2000; Kharevych et al. 2006] Simple interpretation: tradeoff between maintaining velocity, reducing U, and reducing R 18

19 SOME EXISTING MODELS Alignment behavior in boids [Reynolds 1987]: f ij = w x ij v ij R ij = w x ij v ij 2 Velocity obstacles [van den Berg et al 2011]: v ij VO x ij R ij = ቊ if v ij VO x ij 0 otherwise Power law model [Karamouzas et al 2014]: R ij τ(x ij, v ij ) p 19

20 SOME EXISTING MODELS Alignment behavior in boids [Reynolds 1987]: f ij = w x ij v ij R ij = w x ij v ij 2 x j Velocity obstacles [Fiorini and Shiller 1998]: v ij VO x ij R ij = ቊ if v ij VO x ij 0 otherwise v y VO Power law model [Karamouzas et al 2014]: R ij τ(x ij, v ij ) p v x x i 20

21 SOME EXISTING MODELS Alignment behavior in boids [Reynolds 1987]: f ij = w x ij v ij R ij = w x ij v ij 2 Velocity obstacles [Fiorini and Shiller 1998]: v ij VO x ij R ij = ቊ if v ij VO x ij 0 otherwise What is R ij for humans? Power law model [Karamouzas et al 2014]

22 SOME EXISTING MODELS Alignment behavior in boids [Reynolds 1987]: f ij = w x ij v ij R ij = w x ij v ij 2 Velocity obstacles [Fiorini and Shiller 1998]: v ij VO x ij R ij = ቊ if v ij VO x ij 0 otherwise What is R ij for humans? Power law model [Karamouzas et al 2014]

23 II. IMPLICIT CROWDS USING THE POWER-LAW MODEL 23

24 POWER-LAW MODEL [Karamouzas et al. 2014] For each pair of agents: Compute time to collision τ x, v Compute potential R(x, v) τ x, v p Collisions occurs when x ij + v ij τ = r i + r j v i v j x i x j 24

25 POWER-LAW MODEL [Karamouzas et al. 2014] For each pair of agents: Compute time to collision τ x, v Compute potential R(x, v) τ x, v p Collisions occurs when x ij + v ij τ = r i + r j v i v j x i x j 25

26 POWER-LAW MODEL [Karamouzas et al. 2014] For each pair of agents: Compute time to collision τ x, v Compute potential R(x, v) τ x, v p Collisions occurs when x ij + v ij τ = r i + r j v i v j x i x j 26

27 POWER-LAW MODEL [Karamouzas et al. 2014] For each pair of agents: Compute time to collision τ x, v Compute potential R(x, v) τ x, v p Collisions occurs when x ij + v ij τ = r i + r j v i v j x i x j 27

28 IMPLICIT POWER-LAW CROWDS Problem: Apply power law potential to optimization-based backward Euler Easy? Not quite R is discontinuous at boundary of collision cone R becomes infinitely steep as agents graze past R Both phenomena cause numerical solvers to get stuck 28

29 IMPLICIT POWER-LAW CROWDS Problem: Apply power law potential to optimization-based backward Euler Easy? Not quite R is discontinuous at boundary of collision cone R becomes infinitely steep as agents graze past Both phenomena cause numerical solvers to get stuck. 29

30 A CONTINUOUS TTC POTENTIAL Discontinuity due to time to collision (finite if collision predicted, infinite if not) x j x i 30

31 A CONTINUOUS TTC POTENTIAL Discontinuity due to time to collision (finite if collision predicted, infinite if not) x j v p 0 v t x i 31

32 A CONTINUOUS TTC POTENTIAL Discontinuity due to time to collision (finite if collision predicted, infinite if not) x j R v p 0 v t 32

33 A CONTINUOUS TTC POTENTIAL Solution: Let s work with 1 τ (or, imminence of collision) Replace it with a continuous approximation, e.g., by linear extrapolation R becomes C p 1 -smooth 1/τ 33

34 A CONTINUOUS TTC POTENTIAL Solution: Let s work with 1 τ (or, imminence of collision) Replace it with a continuous approximation, e.g., by linear extrapolation R becomes C p 1 -smooth 1/τ 34

35 MAINTAINING SEPARATION Grazing trajectories make R badly behaved Add some distance-based repulsion U ij 1 x ij r ij Continuous collision detection: replace distance x ij with minimum distance over the time step Alternative approach to repulsion: add uncertainty to time to collision computation [Forootaninia 2017] 35

36 MAINTAINING SEPARATION Grazing trajectories make R badly behaved Add some distance-based repulsion U ij 1 x ij r ij Continuous collision detection: replace distance x ij with minimum distance over the time step Alternative approach to repulsion: add uncertainty to time-to-collision computation [Forootaninia et al. 2017] 36

37 III. ANALYSIS AND RESULTS 37

38 THEORETICAL ANALYSIS Implicit integration + continuous PowerLaw potential Guaranteed collision-free motion Smooth (C 2 -continuous) trajectories 38

39 THEORETICAL ANALYSIS Implicit integration + continuous PowerLaw potential Guaranteed collision-free motion Smooth (C 2 -continuous) trajectories Collision-free proof v n+1 minimizes 1 2 vn+1 v M 2 + U(x n+1 ) + R(x n+1, v n+1 ) t R is infinite for a colliding state. U is infinite for a tunneling step. So these cannot be minima. 39

40 COLLISION-FREE MOTION Comparisons to ORCA [van den Berg et al. 2011] (representative velocity-based approach) PowerLaw [Karamouzas et al. 2014] (non-continuous TTC + forward Euler) 40

41 COLLISION-FREE MOTION Comparisons to ORCA [van den Berg et al. 2011] (representative velocity-based approach) PowerLaw [Karamouzas et al. 2014] (non-continuous TTC + forward Euler) 41

42 COLLISION-FREE MOTION Comparisons to ORCA [van den Berg et al. 2011] (representative velocity-based approach) PowerLaw [Karamouzas et al. 2014] (non-continuous TTC + forward Euler) 42

43 FIDELITY TO HUMAN DATA Comparison to human crowds [Charalambous et al. 2014] Implicit Δt=0.4s ORCA Δt=0.4s 43

44 TIME STEP STABILITY (1.5x playback speed) Implicit Δt=0.4s 44

45 TIME STEP STABILITY Motion doesn t change significantly with time step Potential applications: Collision avoidance synced with footstep-based character animation Crowd timestep level of detail without affecting behavior 45

46 PERFORMANCE [fig: performance graph] Δ Δ Cost is linear in number of agents, increases slowly with time step size (Still, 2x-10x slower than ORCA or PowerLaw on a 6-core Intel Xeon E5-1650) Future work: Improve performance via local-global alternating minimization techniques [Narain et al. 2016] [Liu et al. 2017] 46

47 LIMITATIONS AND FUTURE WORK Can other recent crowd models be formulated via interaction energies [Wolinksi et al. 2016, Dutra et al. 2017; ]? Incorporating asymmetrical interactions, e.g., leader-following behavior What is the Δt threshold where quality is maintained? Δt=0.1s Δt=1s Δt=4s 47

48 LIMITATIONS AND FUTURE WORK Can other recent crowd models be formulated via interaction energies [Wolinksi et al. 2016, Dutra et al. 2017; ]? Incorporating asymmetrical interactions, e.g., leader-following behavior What is the Δt threshold where quality is maintained? Δt=0.1s Δt=1s Δt=4s 48

49 FUTURE WORK Applications to LOD systems and footstep-based animation engines Applications to nonlinear dissipation forces in physics-based animation [Xu and Barbic 2017] [Zhu et al. 2015] 49

50 THANK YOU 50

FORCE-BASED BOOKKEEPING

FORCE-BASED BOOKKEEPING LOCAL NAVIGATION 2 FORCE-BASED BOOKKEEPING Social force models The forces are first-class abstractions Agents are considered to be mass particles Other models use forces as bookkeeping It is merely a way

More information

Uncertainty Models for TTC-Based Collision Avoidance

Uncertainty Models for TTC-Based Collision Avoidance Robotics: Science and Systems 2017 Cambridge, MA, USA, July 12-16, 2017 Uncertainty Models for TTC-Based Collision Avoidance Zahra Forootaninia, Ioannis Karamouzas, and Rahul Narain Department of Computer

More information

LOCAL NAVIGATION. Dynamic adaptation of global plan to local conditions A.K.A. local collision avoidance and pedestrian models

LOCAL NAVIGATION. Dynamic adaptation of global plan to local conditions A.K.A. local collision avoidance and pedestrian models LOCAL NAVIGATION 1 LOCAL NAVIGATION Dynamic adaptation of global plan to local conditions A.K.A. local collision avoidance and pedestrian models 2 LOCAL NAVIGATION Why do it? Could we use global motion

More information

Lecture 4: Perceptrons and Multilayer Perceptrons

Lecture 4: Perceptrons and Multilayer Perceptrons Lecture 4: Perceptrons and Multilayer Perceptrons Cognitive Systems II - Machine Learning SS 2005 Part I: Basic Approaches of Concept Learning Perceptrons, Artificial Neuronal Networks Lecture 4: Perceptrons

More information

Iterative Rigid Multibody Dynamics A Comparison of Computational Methods

Iterative Rigid Multibody Dynamics A Comparison of Computational Methods Iterative Rigid Multibody Dynamics A Comparison of Computational Methods Tobias Preclik, Klaus Iglberger, Ulrich Rüde University Erlangen-Nuremberg Chair for System Simulation (LSS) July 1st 2009 T. Preclik

More information

Self-Organization Hypothesis. Lecture 18

Self-Organization Hypothesis. Lecture 18 Self-Organization Hypothesis Lecture 18 Simple attraction & repulsion rules generate schooling behavior positive feedback: brings individuals together negative feedback: but not too close Rules rely on

More information

Gross Motion Planning

Gross Motion Planning Gross Motion Planning...given a moving object, A, initially in an unoccupied region of freespace, s, a set of stationary objects, B i, at known locations, and a legal goal position, g, find a sequence

More information

Introduction to Natural Computation. Lecture 9. Multilayer Perceptrons and Backpropagation. Peter Lewis

Introduction to Natural Computation. Lecture 9. Multilayer Perceptrons and Backpropagation. Peter Lewis Introduction to Natural Computation Lecture 9 Multilayer Perceptrons and Backpropagation Peter Lewis 1 / 25 Overview of the Lecture Why multilayer perceptrons? Some applications of multilayer perceptrons.

More information

Dynamics and Time Integration. Computer Graphics CMU /15-662, Fall 2016

Dynamics and Time Integration. Computer Graphics CMU /15-662, Fall 2016 Dynamics and Time Integration Computer Graphics CMU 15-462/15-662, Fall 2016 Last Quiz: IK (and demo) Last few classes Keyframing and motion capture - Q: what can one NOT do with these techniques? The

More information

Numerical Optimization Algorithms

Numerical Optimization Algorithms Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms

More information

Sections 18.6 and 18.7 Analysis of Artificial Neural Networks

Sections 18.6 and 18.7 Analysis of Artificial Neural Networks Sections 18.6 and 18.7 Analysis of Artificial Neural Networks CS4811 - Artificial Intelligence Nilufer Onder Department of Computer Science Michigan Technological University Outline Univariate regression

More information

CSC242: Intro to AI. Lecture 21

CSC242: Intro to AI. Lecture 21 CSC242: Intro to AI Lecture 21 Administrivia Project 4 (homeworks 18 & 19) due Mon Apr 16 11:59PM Posters Apr 24 and 26 You need an idea! You need to present it nicely on 2-wide by 4-high landscape pages

More information

Neural networks. Chapter 20, Section 5 1

Neural networks. Chapter 20, Section 5 1 Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of

More information

Neural Networks. Nicholas Ruozzi University of Texas at Dallas

Neural Networks. Nicholas Ruozzi University of Texas at Dallas Neural Networks Nicholas Ruozzi University of Texas at Dallas Handwritten Digit Recognition Given a collection of handwritten digits and their corresponding labels, we d like to be able to correctly classify

More information

A Statistical Similarity Measure for Aggregate Crowd Dynamics

A Statistical Similarity Measure for Aggregate Crowd Dynamics A Statistical Similarity Measure for Aggregate Crowd Dynamics Stephen J. Guy UNC-Chapel Hill Jur van den Berg University of Utah Wenxi Liu, Rynson Lau City University of Hong Kong Ming C. Lin, Dinesh Manocha

More information

A Statistical Similarity Measure for Aggregate Crowd Dynamics

A Statistical Similarity Measure for Aggregate Crowd Dynamics A Statistical Similarity Measure for Aggregate Crowd Dynamics Stephen J. Guy UNC-Chapel Hill Jur van den Berg University of Utah Wenxi Liu, Rynson Lau City University of Hong Kong Ming C. Lin, Dinesh Manocha

More information

Neural networks. Chapter 19, Sections 1 5 1

Neural networks. Chapter 19, Sections 1 5 1 Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10

More information

Motion Estimation (I)

Motion Estimation (I) Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

Passive Dynamics and Particle Systems COS 426

Passive Dynamics and Particle Systems COS 426 Passive Dynamics and Particle Systems COS 426 Computer Animation Animation Make objects change over time according to scripted actions Simulation / dynamics Predict how objects change over time according

More information

Multilayer Perceptrons and Backpropagation

Multilayer Perceptrons and Backpropagation Multilayer Perceptrons and Backpropagation Informatics 1 CG: Lecture 7 Chris Lucas School of Informatics University of Edinburgh January 31, 2017 (Slides adapted from Mirella Lapata s.) 1 / 33 Reading:

More information

Ordinary Differential Equations I

Ordinary Differential Equations I Ordinary Differential Equations I CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Ordinary Differential Equations I 1 / 27 Theme of Last Three

More information

Implicit Scheme for the Heat Equation

Implicit Scheme for the Heat Equation Implicit Scheme for the Heat Equation Implicit scheme for the one-dimensional heat equation Once again we consider the one-dimensional heat equation where we seek a u(x, t) satisfying u t = νu xx + f(x,

More information

2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks. Todd W. Neller

2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks. Todd W. Neller 2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks Todd W. Neller Machine Learning Learning is such an important part of what we consider "intelligence" that

More information

Recent Advances in Consensus of Multi-Agent Systems

Recent Advances in Consensus of Multi-Agent Systems Int. Workshop on Complex Eng. Systems and Design, Hong Kong, 22 May 2009 Recent Advances in Consensus of Multi-Agent Systems Jinhu Lü Academy of Mathematics and Systems Science Chinese Academy of Sciences

More information

Modeling, Simulating and Rendering Fluids. Thanks to Ron Fediw et al, Jos Stam, Henrik Jensen, Ryan

Modeling, Simulating and Rendering Fluids. Thanks to Ron Fediw et al, Jos Stam, Henrik Jensen, Ryan Modeling, Simulating and Rendering Fluids Thanks to Ron Fediw et al, Jos Stam, Henrik Jensen, Ryan Applications Mostly Hollywood Shrek Antz Terminator 3 Many others Games Engineering Animating Fluids is

More information

Optimization and Gradient Descent

Optimization and Gradient Descent Optimization and Gradient Descent INFO-4604, Applied Machine Learning University of Colorado Boulder September 12, 2017 Prof. Michael Paul Prediction Functions Remember: a prediction function is the function

More information

Sections 18.6 and 18.7 Artificial Neural Networks

Sections 18.6 and 18.7 Artificial Neural Networks Sections 18.6 and 18.7 Artificial Neural Networks CS4811 - Artificial Intelligence Nilufer Onder Department of Computer Science Michigan Technological University Outline The brain vs artifical neural networks

More information

Neural networks. Chapter 20. Chapter 20 1

Neural networks. Chapter 20. Chapter 20 1 Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms

More information

Motion Estimation (I) Ce Liu Microsoft Research New England

Motion Estimation (I) Ce Liu Microsoft Research New England Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence Jeff Clune Assistant Professor Evolving Artificial Intelligence Laboratory Announcements Be making progress on your projects! Three Types of Learning Unsupervised Supervised Reinforcement

More information

Introduction to multiscale modeling and simulation. Explicit methods for ODEs : forward Euler. y n+1 = y n + tf(y n ) dy dt = f(y), y(0) = y 0

Introduction to multiscale modeling and simulation. Explicit methods for ODEs : forward Euler. y n+1 = y n + tf(y n ) dy dt = f(y), y(0) = y 0 Introduction to multiscale modeling and simulation Lecture 5 Numerical methods for ODEs, SDEs and PDEs The need for multiscale methods Two generic frameworks for multiscale computation Explicit methods

More information

Sections 18.6 and 18.7 Artificial Neural Networks

Sections 18.6 and 18.7 Artificial Neural Networks Sections 18.6 and 18.7 Artificial Neural Networks CS4811 - Artificial Intelligence Nilufer Onder Department of Computer Science Michigan Technological University Outline The brain vs. artifical neural

More information

ANALYSIS OF CONSENSUS AND COLLISION AVOIDANCE USING THE COLLISION CONE APPROACH IN THE PRESENCE OF TIME DELAYS. A Thesis by. Dipendra Khatiwada

ANALYSIS OF CONSENSUS AND COLLISION AVOIDANCE USING THE COLLISION CONE APPROACH IN THE PRESENCE OF TIME DELAYS. A Thesis by. Dipendra Khatiwada ANALYSIS OF CONSENSUS AND COLLISION AVOIDANCE USING THE COLLISION CONE APPROACH IN THE PRESENCE OF TIME DELAYS A Thesis by Dipendra Khatiwada Bachelor of Science, Wichita State University, 2013 Submitted

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21

Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA   1/ 21 Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural

More information

Neural Networks in Structured Prediction. November 17, 2015

Neural Networks in Structured Prediction. November 17, 2015 Neural Networks in Structured Prediction November 17, 2015 HWs and Paper Last homework is going to be posted soon Neural net NER tagging model This is a new structured model Paper - Thursday after Thanksgiving

More information

Time scales. Notes. Mixed Implicit/Explicit. Newmark Methods [ ] [ ] "!t = O 1 %

Time scales. Notes. Mixed Implicit/Explicit. Newmark Methods [ ] [ ] !t = O 1 % Notes Time scales! For using Pixie (the renderer) make sure you type use pixie first! Assignment 1 questions?! [work out]! For position dependence, characteristic time interval is "!t = O 1 % $ ' # K &!

More information

AI Programming CS F-20 Neural Networks

AI Programming CS F-20 Neural Networks AI Programming CS662-2008F-20 Neural Networks David Galles Department of Computer Science University of San Francisco 20-0: Symbolic AI Most of this class has been focused on Symbolic AI Focus or symbols

More information

An Overview of Fluid Animation. Christopher Batty March 11, 2014

An Overview of Fluid Animation. Christopher Batty March 11, 2014 An Overview of Fluid Animation Christopher Batty March 11, 2014 What distinguishes fluids? What distinguishes fluids? No preferred shape. Always flows when force is applied. Deforms to fit its container.

More information

A Space-Time Expansion Discontinuous Galerkin Scheme with Local Time-Stepping for the Ideal and Viscous MHD Equations

A Space-Time Expansion Discontinuous Galerkin Scheme with Local Time-Stepping for the Ideal and Viscous MHD Equations A Space-Time Expansion Discontinuous Galerkin Scheme with Local Time-Stepping for the Ideal and Viscous MHD Equations Ch. Altmann, G. Gassner, F. Lörcher, C.-D. Munz Numerical Flow Models for Controlled

More information

On Multigrid for Phase Field

On Multigrid for Phase Field On Multigrid for Phase Field Carsten Gräser (FU Berlin), Ralf Kornhuber (FU Berlin), Rolf Krause (Uni Bonn), and Vanessa Styles (University of Sussex) Interphase 04 Rome, September, 13-16, 2004 Synopsis

More information

Ordinary Differential Equations I

Ordinary Differential Equations I Ordinary Differential Equations I CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Ordinary Differential Equations I 1 / 32 Theme of Last Few

More information

Dynamics and Time Integration

Dynamics and Time Integration Lecture 17: Dynamics and Time Integration Computer Graphics CMU 15-462/15-662, Fall 2015 Last time: animation Added motion to our model Interpolate keyframes Still a lot of work! Today: physically-based

More information

100 inference steps doesn't seem like enough. Many neuron-like threshold switching units. Many weighted interconnections among units

100 inference steps doesn't seem like enough. Many neuron-like threshold switching units. Many weighted interconnections among units Connectionist Models Consider humans: Neuron switching time ~ :001 second Number of neurons ~ 10 10 Connections per neuron ~ 10 4 5 Scene recognition time ~ :1 second 100 inference steps doesn't seem like

More information

Convolutional networks. Sebastian Seung

Convolutional networks. Sebastian Seung Convolutional networks Sebastian Seung Convolutional network Neural network with spatial organization every neuron has a location usually on a grid Translation invariance synaptic strength depends on locations

More information

y(x n, w) t n 2. (1)

y(x n, w) t n 2. (1) Network training: Training a neural network involves determining the weight parameter vector w that minimizes a cost function. Given a training set comprising a set of input vector {x n }, n = 1,...N,

More information

Parallel-in-time integrators for Hamiltonian systems

Parallel-in-time integrators for Hamiltonian systems Parallel-in-time integrators for Hamiltonian systems Claude Le Bris ENPC and INRIA Visiting Professor, The University of Chicago joint work with X. Dai (Paris 6 and Chinese Academy of Sciences), F. Legoll

More information

Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice

Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice 1 Lecture Notes, HCI, 4.1.211 Chapter 2 Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice Bastian Goldlücke Computer Vision Group Technical University of Munich 2 Bastian

More information

Procedural Animation. D.A. Forsyth

Procedural Animation. D.A. Forsyth Procedural Animation D.A. Forsyth Big points Two important types of procedural animation slice and dice data, like texture synthesis build (approximate) physical simulation Extremely powerful issues how

More information

Introduction to Machine Learning Spring 2018 Note Neural Networks

Introduction to Machine Learning Spring 2018 Note Neural Networks CS 189 Introduction to Machine Learning Spring 2018 Note 14 1 Neural Networks Neural networks are a class of compositional function approximators. They come in a variety of shapes and sizes. In this class,

More information

Last update: October 26, Neural networks. CMSC 421: Section Dana Nau

Last update: October 26, Neural networks. CMSC 421: Section Dana Nau Last update: October 26, 207 Neural networks CMSC 42: Section 8.7 Dana Nau Outline Applications of neural networks Brains Neural network units Perceptrons Multilayer perceptrons 2 Example Applications

More information

18.6 Regression and Classification with Linear Models

18.6 Regression and Classification with Linear Models 18.6 Regression and Classification with Linear Models 352 The hypothesis space of linear functions of continuous-valued inputs has been used for hundreds of years A univariate linear function (a straight

More information

Lecture 38 Insulated Boundary Conditions

Lecture 38 Insulated Boundary Conditions Lecture 38 Insulated Boundary Conditions Insulation In many of the previous sections we have considered fixed boundary conditions, i.e. u(0) = a, u(l) = b. We implemented these simply by assigning u j

More information

Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks Facultés Universitaires Notre-Dame de la Paix 27 March 2007 Outline 1 Introduction 2 Fundamentals Biological neuron Artificial neuron Artificial Neural Network Outline 3 Single-layer ANN Perceptron Adaline

More information

Particle Systems. University of Texas at Austin CS384G - Computer Graphics

Particle Systems. University of Texas at Austin CS384G - Computer Graphics Particle Systems University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell Reading Required: Witkin, Particle System Dynamics, SIGGRAPH 97 course notes on Physically Based Modeling.

More information

Back-Propagation Algorithm. Perceptron Gradient Descent Multilayered neural network Back-Propagation More on Back-Propagation Examples

Back-Propagation Algorithm. Perceptron Gradient Descent Multilayered neural network Back-Propagation More on Back-Propagation Examples Back-Propagation Algorithm Perceptron Gradient Descent Multilayered neural network Back-Propagation More on Back-Propagation Examples 1 Inner-product net =< w, x >= w x cos(θ) net = n i=1 w i x i A measure

More information

Waves in a Shock Tube

Waves in a Shock Tube Waves in a Shock Tube Ivan Christov c February 5, 005 Abstract. This paper discusses linear-wave solutions and simple-wave solutions to the Navier Stokes equations for an inviscid and compressible fluid

More information

Numerical methods for the Navier- Stokes equations

Numerical methods for the Navier- Stokes equations Numerical methods for the Navier- Stokes equations Hans Petter Langtangen 1,2 1 Center for Biomedical Computing, Simula Research Laboratory 2 Department of Informatics, University of Oslo Dec 6, 2012 Note:

More information

Part IV: Numerical schemes for the phase-filed model

Part IV: Numerical schemes for the phase-filed model Part IV: Numerical schemes for the phase-filed model Jie Shen Department of Mathematics Purdue University IMS, Singapore July 29-3, 29 The complete set of governing equations Find u, p, (φ, ξ) such that

More information

Random walks for deformable image registration Dana Cobzas

Random walks for deformable image registration Dana Cobzas Random walks for deformable image registration Dana Cobzas with Abhishek Sen, Martin Jagersand, Neil Birkbeck, Karteek Popuri Computing Science, University of Alberta, Edmonton Deformable image registration?t

More information

Stabilizing Integrators for Real-Time Physics

Stabilizing Integrators for Real-Time Physics 9 Stabilizing Integrators for Real-Time Physics DIMITAR DINEV, University of Utah TIANTIAN LIU, University of Pennsylvania LADISLAV KAVAN, University of Utah We present a new time integration method featuring

More information

Linear Regression. CSL603 - Fall 2017 Narayanan C Krishnan

Linear Regression. CSL603 - Fall 2017 Narayanan C Krishnan Linear Regression CSL603 - Fall 2017 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis Regularization

More information

Linear Regression. CSL465/603 - Fall 2016 Narayanan C Krishnan

Linear Regression. CSL465/603 - Fall 2016 Narayanan C Krishnan Linear Regression CSL465/603 - Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis

More information

Introduction to Scientific Computing

Introduction to Scientific Computing Introduction to Scientific Computing Benson Muite benson.muite@ut.ee http://kodu.ut.ee/ benson https://courses.cs.ut.ee/218/isc/spring 5 March 218 [Public Domain, https://commons.wikimedia.org/wiki/file:curie

More information

4. Multilayer Perceptrons

4. Multilayer Perceptrons 4. Multilayer Perceptrons This is a supervised error-correction learning algorithm. 1 4.1 Introduction A multilayer feedforward network consists of an input layer, one or more hidden layers, and an output

More information

Math background. Physics. Simulation. Related phenomena. Frontiers in graphics. Rigid fluids

Math background. Physics. Simulation. Related phenomena. Frontiers in graphics. Rigid fluids Fluid dynamics Math background Physics Simulation Related phenomena Frontiers in graphics Rigid fluids Fields Domain Ω R2 Scalar field f :Ω R Vector field f : Ω R2 Types of derivatives Derivatives measure

More information

Lecture 41: Highlights

Lecture 41: Highlights Lecture 41: Highlights The goal of this lecture is to remind you of some of the key points that we ve covered this semester Note that this is not the complete set of topics that may appear on the final

More information

Fluid Animation. Christopher Batty November 17, 2011

Fluid Animation. Christopher Batty November 17, 2011 Fluid Animation Christopher Batty November 17, 2011 What distinguishes fluids? What distinguishes fluids? No preferred shape Always flows when force is applied Deforms to fit its container Internal forces

More information

ARock: an algorithmic framework for asynchronous parallel coordinate updates

ARock: an algorithmic framework for asynchronous parallel coordinate updates ARock: an algorithmic framework for asynchronous parallel coordinate updates Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin ( UCLA Math, U.Waterloo DCO) UCLA CAM Report 15-37 ShanghaiTech SSDS 15 June 25,

More information

Machine Learning Applied to 3-D Reservoir Simulation

Machine Learning Applied to 3-D Reservoir Simulation Machine Learning Applied to 3-D Reservoir Simulation Marco A. Cardoso 1 Introduction The optimization of subsurface flow processes is important for many applications including oil field operations and

More information

CS598 Machine Learning in Computational Biology (Lecture 5: Matrix - part 2) Professor Jian Peng Teaching Assistant: Rongda Zhu

CS598 Machine Learning in Computational Biology (Lecture 5: Matrix - part 2) Professor Jian Peng Teaching Assistant: Rongda Zhu CS598 Machine Learning in Computational Biology (Lecture 5: Matrix - part 2) Professor Jian Peng Teaching Assistant: Rongda Zhu Feature engineering is hard 1. Extract informative features from domain knowledge

More information

Machine Learning: Multi Layer Perceptrons

Machine Learning: Multi Layer Perceptrons Machine Learning: Multi Layer Perceptrons Prof. Dr. Martin Riedmiller Albert-Ludwigs-University Freiburg AG Maschinelles Lernen Machine Learning: Multi Layer Perceptrons p.1/61 Outline multi layer perceptrons

More information

PDE Solvers for Fluid Flow

PDE Solvers for Fluid Flow PDE Solvers for Fluid Flow issues and algorithms for the Streaming Supercomputer Eran Guendelman February 5, 2002 Topics Equations for incompressible fluid flow 3 model PDEs: Hyperbolic, Elliptic, Parabolic

More information

CS:4420 Artificial Intelligence

CS:4420 Artificial Intelligence CS:4420 Artificial Intelligence Spring 2018 Neural Networks Cesare Tinelli The University of Iowa Copyright 2004 18, Cesare Tinelli and Stuart Russell a a These notes were originally developed by Stuart

More information

Exam in TMA4215 December 7th 2012

Exam in TMA4215 December 7th 2012 Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Contact during the exam: Elena Celledoni, tlf. 7359354, cell phone 48238584 Exam in TMA425 December 7th 22 Allowed

More information

Chapter 1. Introduction

Chapter 1. Introduction Chapter 1 Introduction Many astrophysical scenarios are modeled using the field equations of fluid dynamics. Fluids are generally challenging systems to describe analytically, as they form a nonlinear

More information

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Neural Networks Varun Chandola x x 5 Input Outline Contents February 2, 207 Extending Perceptrons 2 Multi Layered Perceptrons 2 2. Generalizing to Multiple Labels.................

More information

q t = F q x. (1) is a flux of q due to diffusion. Although very complex parameterizations for F q

q t = F q x. (1) is a flux of q due to diffusion. Although very complex parameterizations for F q ! Revised Tuesday, December 8, 015! 1 Chapter 7: Diffusion Copyright 015, David A. Randall 7.1! Introduction Diffusion is a macroscopic statistical description of microscopic advection. Here microscopic

More information

Computational statistics

Computational statistics Computational statistics Lecture 3: Neural networks Thierry Denœux 5 March, 2016 Neural networks A class of learning methods that was developed separately in different fields statistics and artificial

More information

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University Final Examination CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University The exam runs for 3 hours. The exam contains eight problems. You must complete the first

More information

A Generalized Maximum Dissipation Principle in an Impulse-velocity Time-stepping Scheme

A Generalized Maximum Dissipation Principle in an Impulse-velocity Time-stepping Scheme A Generalized Maximum Dissipation Principle in an Impulse-velocity Time-stepping Scheme T. Preclik, U. Rüde September 2, 213 Chair of Computer Science 1 (System Simulation) University of Erlangen-Nürnberg,

More information

Neural Networks biological neuron artificial neuron 1

Neural Networks biological neuron artificial neuron 1 Neural Networks biological neuron artificial neuron 1 A two-layer neural network Output layer (activation represents classification) Weighted connections Hidden layer ( internal representation ) Input

More information

Linear Models for Regression CS534

Linear Models for Regression CS534 Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict

More information

Artificial Neural Networks

Artificial Neural Networks Artificial Neural Networks Threshold units Gradient descent Multilayer networks Backpropagation Hidden layer representations Example: Face Recognition Advanced topics 1 Connectionist Models Consider humans:

More information

Learning Energy-Based Models of High-Dimensional Data

Learning Energy-Based Models of High-Dimensional Data Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal

More information

Implicit Euler Numerical Simulation of Sliding Mode Systems

Implicit Euler Numerical Simulation of Sliding Mode Systems Implicit Euler Numerical Simulation of Sliding Mode Systems Vincent Acary and Bernard Brogliato, INRIA Grenoble Rhône-Alpes Conference in honour of E. Hairer s 6th birthday Genève, 17-2 June 29 Objectives

More information

Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels

Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)

More information

Artificial Neural Networks. Edward Gatt

Artificial Neural Networks. Edward Gatt Artificial Neural Networks Edward Gatt What are Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning Very

More information

Erkut Erdem. Hacettepe University February 24 th, Linear Diffusion 1. 2 Appendix - The Calculus of Variations 5.

Erkut Erdem. Hacettepe University February 24 th, Linear Diffusion 1. 2 Appendix - The Calculus of Variations 5. LINEAR DIFFUSION Erkut Erdem Hacettepe University February 24 th, 2012 CONTENTS 1 Linear Diffusion 1 2 Appendix - The Calculus of Variations 5 References 6 1 LINEAR DIFFUSION The linear diffusion (heat)

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning First-Order Methods, L1-Regularization, Coordinate Descent Winter 2016 Some images from this lecture are taken from Google Image Search. Admin Room: We ll count final numbers

More information

Adaptive Primal Dual Optimization for Image Processing and Learning

Adaptive Primal Dual Optimization for Image Processing and Learning Adaptive Primal Dual Optimization for Image Processing and Learning Tom Goldstein Rice University tag7@rice.edu Ernie Esser University of British Columbia eesser@eos.ubc.ca Richard Baraniuk Rice University

More information

Various lecture notes for

Various lecture notes for Various lecture notes for 18311. R. R. Rosales (MIT, Math. Dept., 2-337) April 12, 2013 Abstract Notes, both complete and/or incomplete, for MIT s 18.311 (Principles of Applied Mathematics). These notes

More information

Neural Networks. Hopfield Nets and Auto Associators Fall 2017

Neural Networks. Hopfield Nets and Auto Associators Fall 2017 Neural Networks Hopfield Nets and Auto Associators Fall 2017 1 Story so far Neural networks for computation All feedforward structures But what about.. 2 Loopy network Θ z = ቊ +1 if z > 0 1 if z 0 y i

More information

The method of lines (MOL) for the diffusion equation

The method of lines (MOL) for the diffusion equation Chapter 1 The method of lines (MOL) for the diffusion equation The method of lines refers to an approximation of one or more partial differential equations with ordinary differential equations in just

More information

Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications

Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications Qixing Huang The University of Texas at Austin huangqx@cs.utexas.edu 1 Disclaimer This note is adapted from Section

More information

ESO 208A: Computational Methods in Engineering. Saumyen Guha

ESO 208A: Computational Methods in Engineering. Saumyen Guha ESO 208A: Computational Methods in Engineering Introduction, Error Analysis Saumyen Guha Department of Civil Engineering IIT Kanpur What is Computational Methods or Numerical Methods in Engineering? Formulation

More information

Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations

Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations S. Y. Ha and J. Park Department of Mathematical Sciences Seoul National University Sep 23, 2013 Contents 1 Logistic Map 2 Euler and

More information

19 : Slice Sampling and HMC

19 : Slice Sampling and HMC 10-708: Probabilistic Graphical Models 10-708, Spring 2018 19 : Slice Sampling and HMC Lecturer: Kayhan Batmanghelich Scribes: Boxiang Lyu 1 MCMC (Auxiliary Variables Methods) In inference, we are often

More information

Variational Methods in Image Denoising

Variational Methods in Image Denoising Variational Methods in Image Denoising Jamylle Carter Postdoctoral Fellow Mathematical Sciences Research Institute (MSRI) MSRI Workshop for Women in Mathematics: Introduction to Image Analysis 22 January

More information