A Background Layer Model for Object Tracking through Occlusion

Similar documents
Machine Learning Techniques for Computer Vision

Computational Biology Lecture #3: Probability and Statistics. Bud Mishra Professor of Computer Science, Mathematics, & Cell Biology Sept

Human Pose Tracking I: Basics. David Fleet University of Toronto

Statistical Filters for Crowd Image Analysis

Automated Segmentation of Low Light Level Imagery using Poisson MAP- MRF Labelling

Markov chain Monte Carlo methods for visual tracking

SYDE 372 Introduction to Pattern Recognition. Probability Measures for Classification: Part I

Linear Dynamical Systems (Kalman filter)

Bayesian Machine Learning

Mixture Models and EM

Hidden Markov Models Part 2: Algorithms

Markov Chains and Hidden Markov Models

Naïve Bayes classification

Statistical and Learning Techniques in Computer Vision Lecture 2: Maximum Likelihood and Bayesian Estimation Jens Rittscher and Chuck Stewart

Chapter 3: Maximum-Likelihood & Bayesian Parameter Estimation (part 1)

Notes on Machine Learning for and

Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012

Introduction to Bayesian methods in inverse problems

Bayesian Networks BY: MOHAMAD ALSABBAGH

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions

Naïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability

Lecture : Probabilistic Machine Learning

Statistical learning. Chapter 20, Sections 1 4 1

Master 2 Informatique Probabilistic Learning and Data Analysis

Machine Learning. Gaussian Mixture Models. Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall

Mobile Robot Localization

Bayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models

Brief Introduction of Machine Learning Techniques for Content Analysis

Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a

9 Multi-Model State Estimation

Density Estimation. Seungjin Choi

Machine Learning 4771

Generative Clustering, Topic Modeling, & Bayesian Inference

Probabilistic Robotics

Basic math for biology

Lecture 4: Probabilistic Learning

State Space and Hidden Markov Models

Bayesian Decision Theory

MCMC Sampling for Bayesian Inference using L1-type Priors

CS 4495 Computer Vision Principle Component Analysis

Introduction to Machine Learning

y Xw 2 2 y Xw λ w 2 2

COMP90051 Statistical Machine Learning

COMS 4721: Machine Learning for Data Science Lecture 16, 3/28/2017

Second-Order Polynomial Models for Background Subtraction

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Introduction: MLE, MAP, Bayesian reasoning (28/8/13)

Non-Parametric Bayes

Probabilistic Time Series Classification

Modeling Symmetries for Stochastic Structural Recognition

EM & Variational Bayes

Introduction to Bayesian Learning. Machine Learning Fall 2018

Algorithmisches Lernen/Machine Learning

Intelligent Systems:

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li.

Estimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator

Kalman filtering and friends: Inference in time series models. Herke van Hoof slides mostly by Michael Rubinstein

Lecture 8 Learning Sequence Motif Models Using Expectation Maximization (EM) Colin Dewey February 14, 2008

Recent Advances in Bayesian Inference Techniques

Lecture 6: Gaussian Mixture Models (GMM)

The Expectation-Maximization Algorithm

Gaussian Mixture Models

Abnormal Activity Detection and Tracking Namrata Vaswani Dept. of Electrical and Computer Engineering Iowa State University

Unsupervised Learning

Discriminative Fields for Modeling Spatial Dependencies in Natural Images

Machine Learning CMPT 726 Simon Fraser University. Binomial Parameter Estimation

Introduction into Bayesian statistics

Independent Component Analysis and Unsupervised Learning

Expectation Propagation Algorithm

Performance Comparison of K-Means and Expectation Maximization with Gaussian Mixture Models for Clustering EE6540 Final Project

Introduction to Machine Learning

CS Homework 3. October 15, 2009

CSC487/2503: Foundations of Computer Vision. Visual Tracking. David Fleet

Bayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Mobile Robot Localization

Machine Learning. Classification. Bayes Classifier. Representing data: Choosing hypothesis class. Learning: h:x a Y. Eric Xing

Machine Learning for Signal Processing Bayes Classification and Regression

2D Image Processing (Extended) Kalman and particle filter

A Bayesian Perspective on Residential Demand Response Using Smart Meter Data

Ridge Regression. Mohammad Emtiyaz Khan EPFL Oct 1, 2015

CSC411 Fall 2018 Homework 5

STA 4273H: Statistical Machine Learning

9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering

MIXTURE MODELS AND EM

ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4

Digital Matting. Outline. Introduction to Digital Matting. Introduction to Digital Matting. Compositing equation: C = α * F + (1- α) * B

Graphical Models for Collaborative Filtering

Introduction to Probabilistic Machine Learning

Midterm sample questions

Probabilistic Reasoning in Deep Learning

Learning Gaussian Process Models from Uncertain Data

Statistical learning. Chapter 20, Sections 1 3 1

Probability models for machine learning. Advanced topics ML4bio 2016 Alan Moses

Spatial Statistics with Image Analysis. Outline. A Statistical Approach. Johan Lindström 1. Lund October 6, 2016

Statistical Pattern Recognition

Lecture 3: Machine learning, classification, and generative models

Variational Message Passing. By John Winn, Christopher M. Bishop Presented by Andy Miller

Latent Variable Models

A Study of the Kalman Filter applied to Visual Tracking

p(d θ ) l(θ ) 1.2 x x x

Transcription:

A Background Layer Model for Object Tracking through Occlusion Yue Zhou and Hai Tao UC-Santa Cruz Presented by: Mei-Fang Huang CSE252C class, November 6, 2003

Overview Object tracking problems Dynamic layer model Estimating model parameters Implementation and results Conclusion and discussion

Tracking Problems Goal: Estimate 2D or 3D positions of foreground and background objects over time Approach: Model-based e.g. Condensation Layer-based e.g. Dynamic layer model e.g. Flexible sprites

Dynamic Layer Model Layer model Represent of moving objects with different motion into different layers Layered representation for motion analysis -Wang & Adelson, 1993 CVPR Dynamic Layer Model Dynamic? Components? Estimation? Dynamically Motion update layer model EM algorithm Segmentation for Allow layer order alteration and maximum Appearance a posteriori layer deletion and creation

Contributions of DLM Object tracking with bayesian estimation of dynamic layer representation - H. Tao, H. S. Sawhney, R. Kumar, 2002 IEEE transactions on pattern analysis and machine intelligence Complete representation Dynamic estimation Insufficient in objects with occlusion New ideas in this paper Introduce Ordering Information Z-depth -Foreground / background layer ordering -Allow multiple background layers

New Layer Representation Scene representation with Depth Ordering Information Moving objects are described as Foreground Layers Others belong to Background Occluding Layers Interlace foreground and background layers

Difference from previous work Use multiple background layers instead of only one background layer Try to solve complicated occlusion problems

How does it work? Video frame(t-1) Video frame(t) Layer Representation Estimation Optimize object state Object state(t-1) Layer ordering Motion Shape Object state(t)? Optimal object state(t) Layer ordering Motion Shape Appearance Appearance

Goal We want to estimate: Foreground layer ordering Background layer Motion layer parameters Achieved by: MAP framework (Maximum A Posteriori) arg max P( Λ Λ, I,..., I ) Λ t t t 1 t 0

The MAP estimation arg max P( Λ Λ, I,..., I ) Λ t t t 1 t 0 Λ t t : the state of the tracker at time t I : the image observation at time t

The MAP estimation Using Bayes rule & HMM P( Λ Λ, I,..., I ) = P( I Λ ) P( Λ Λ ) t t 1 t 0 t t t t 1 Λ t t : the state of the tracker at time t I : the image observation at time t PY ( X, E) = PX ( YEPY, ) ( E) PX ( E) arg max P( Λ Λ I,..., I ) Λ t t t 1, t 0 = arg max Λ t P( It,..., I0 Λ t, Λ t 1) P( Λ t Λ t 1) P( I,..., I Λ ) 0 t 1 = arg max P( I Λ ) P( Λ Λ ) Λ t t t t t t 1

Estimation Recall that we have P( Λ Λ, I,..., I ) = P( I Λ ) P( Λ Λ ) t t 1 t 0 t t t t 1 Likelihood Prior

Apply the motion layer models Prior function P( Λ Λ ) = P P P P P where t t 1 order fg _ shape bg _ shape motion appearance P = P ( o o ) order t t 1 L P = P ( τ ( x ) τ ( x )) fg _ sh a p e t, j i t 1, j i j = 1 i = 1 L + 1 P = P ( π ( x ) π ( x )) P bg _ shape t, j i t 1, j i j = 1 i = 1 = L P ( θ θ ) motion t, j t 1, j j = 1 L N N N j j j P = P ( A ( x ) A ( x )) appearance t, j i t 1, j i j = 1 i = 1

Apply the motion layer models Likelihood function N P ( I Λ ) = ( P ( x ) + P ( x )) t t bgo i fgo i i = 1 One background layer/ multiple foreground layer Background observation probability Probability in one background layer P ( x ) = P( I( x ) B( x )) P ( x ) bgo i i i B i L P ( x ) = [ P( I ( x ) A ( x )) P ( x )] fgo i j i j i j i j = 1 jth foreground is visible

Estimate Sub-Problems Approximate solution: Divide it into sub-problems

Motion Layer Analysis Model the shape and appearance of each object Some approaches Gaussian distributions Markov Random Fields Mixture models (Use EM to get weight) Gaussian segmentation prior function Shape map/mask

Motion Layer Parameters Foreground Layer Motion / Shape / Appearance models Parameters: Position, Orientation, Scale Background Layer Shape / Appearance models Shared a single motion Depth ordering Background Layer

What to do if there is occlusion? Depth order back front Layer ordering Layer model Optimal model EM

Implementation of DLM Motion models Shape models Layer visibility Shape dynamics Appearance model

Motion Models Foreground Position Translation + rotation Constant velocity Background Planar projective ω t ( u, v ) t t P( θ θ ) = N( θ : Φθ, Q) t t 1 t t 1 Φ : standard transition matrix for a constant velocity model Θ = [ µ, ω, s, µ, ω, s ]

Implementation of DLM Motion models Shape models Layer visibility Shape dynamics Appearance model

Shape Models Shape map (a priori) Foreground layers Background layers τ i, j( xi) π i, j( xi) Gaussian segmentation prior function Assumption Each pixel only belongs to ONE background layer π i, j( xi) = 1 j

Implementation of DLM Motion models Shape models Layer visibility Shape dynamics Appearance model

Layer Visibility See j-th foreground j j 1 P ( x ) = τ ( x )(1 π ( x )) [1 τ ( x )] j i j i l i s i l= 1 s= 1 τ i, j( xi) π i, j( xi) foreground background In j-th foreground layer shape * not in a background layer * not in 1~(j-1)-th foreground layer See j-th background j 1 P ( x ) = π ( x ) (1 τ ( x )) B, j i j i k i k = 1 In j-th background layer shape * not in 1~(j-1)-th foreground layer Observe one background L + 1 j 1 PB ( x i ) = π j( x i ) (1 τ k ( x i )) j= 1 k = 1 Sum all the possible background layers

Implementation of DLM Motion models Shape models Layer visibility Shape dynamics Appearance model

Shape Dynamics Assumption Shapes don t change dramatically Use constant velocity model Constant value Gaussian model P( τ ( x ) τ ( x )) t, j i t 1, j i = γ + N τ x τ R ω x µ s στ 2 ( t, j( i) : t 1, j( ( t, j)( i t, j) / t, j), ) Shape map alignment

Implementation of DLM Motion models Shape models Layer visibility Shape dynamics Appearance model

Appearance Models A t, j (x i ) Constant value over time Image model is a Gaussian distribution with A t The temporal change of A t is also a Guassian distribution PI x A x NI x A x 2 ( t( i) t, j( i)) = ( t( i): t, j( i), σ I ) PA x A x NA x A x 2 ( t, j( i) t 1, j( i)) = ( t, j( i): t 1, j( i), A) σ

Summary: State estimation P( Λ Λ, I,..., I ) t t 1 t 0 = P P P P P ( P ( x ) + P ( x )) order fg _ shape bg _ shape motion appearance bgo i fg o i i = 1 STEP 1: Find layer ordering Go through all possible orderings and maximize the posterior probability STEP 2: Motion estimation Relaxing problem to ( P ( x ) + P ( x )) P i= 1 STEP 3: Foreground shape STEP 4: Background shape STEP 5: Appearance estimation n bgo i fgo i motion Search appearance value between current observation and the previous estimate N

Results Foreground shapes Background shapes Foreground appearance

Results

Conclusion Achievement Handling difficult occlusion problem Solving occlusion caused by the foreground and background objects Future work Efficient optimization algorithms for optimal foreground layer ordering Flexible shape and motion model

Comparison Moving Objects Background Model parameters Estimation Flexible Sprites Sprite layer One layer Sprite mask Appearance EM algorithm Background Layer Model Foreground layer Multiple layer Depth order, motion shape, appearance EM for MAP + HMM

Discussion Why use background layers? Can we just take the background layers as foreground layers? When will this approach fail? Under which condition? Observation noisy (Synthetic video) Motion blur Does it work when there are multiple views?

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback