Lecture 13: Tracking mo3on features op3cal flow
|
|
|
- Vivien Dawson
- 5 years ago
- Views:
Transcription
1 Lecture 13: Tracking mo3on features op3cal flow Professor Fei- Fei Li Stanford Vision Lab Lecture 14-1!
2 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons Reading: [Szeliski] Chapters: 8.4, 8.5 Lecture 14-2!
3 From images to videos A video is a sequence of frames captured over 3me Now our image data is a func3on of space (x, y) and 3me (t) Lecture 14-3!
4 Mo3on es3ma3on techniques Op3cal flow Recover image mo3on at each pixel from spa3o- temporal image brightness varia3ons (op3cal flow) Feature- tracking Extract visual features (corners, textured areas) and track them over mul3ple frames Lecture 14-4!
5 Op3cal flow Vector field func3on of the spa3o- temporal image brightness varia3ons Picture courtesy of Selim Temizer - Learning and Intelligent Systems (LIS) Group, MIT Lecture 14-5!
6 Feature- tracking Courtesy of Jean- Yves Bouguet Vision Lab, California Ins3tute of Technology Lecture 14-6!
7 Feature- tracking Courtesy of Jean- Yves Bouguet Vision Lab, California Ins3tute of Technology Lecture 14-7!
8 Op3cal flow Defini3on: op3cal flow is the apparent mo3on of brightness pagerns in the image Note: apparent mo3on can be caused by ligh3ng changes without any actual mo3on Think of a uniform rota3ng sphere under fixed ligh3ng vs. a sta3onary sphere under moving illumina3on GOAL: Recover image mo3on at each pixel from op3cal flow Lecture 14-8!
9 Es3ma3ng op3cal flow I(x,y,t 1) I(x,y,t) Given two subsequent frames, es3mate the apparent mo3on field u(x,y), v(x,y) between them Key assump3ons Brightness constancy: projec3on of the same point looks the same in every frame Small mo9on: points do not move very far Spa9al coherence: points move like their neighbors Lecture 14-9!
10 Lecture 14 -! t y x I y x v I y x u I t y x I t u y u x I ), ( ), ( 1),, ( ),, ( Brightness Constancy Equa3on: ), ( 1),, ( ),, ( ), ( t y x y x v y u x I t y x I + + = Linearizing the right side using Taylor expansion: The brightness constancy constraint I(x,y,t 1) I(x,y,t) t y x I v I u I Hence, Image deriva3ve along x [ ] 0 I v u I t T = + t y x I y x v I y x u I t y x I t u y u x I + + = + + ), ( ), ( 1),, ( ),, ( 10
11 The brightness constancy constraint Can we use this equa3on to recover image mo3on (u,v) at each pixel? I [ ] T u v + I = 0 How many equa3ons and unknowns per pixel? One equa3on (this is a scalar equa3on!), two unknowns (u,v) t The component of the flow perpendicular to the gradient (i.e., parallel to the edge) cannot be measured If (u, v ) sa3sfies the equa3on, so does (u+u, v+v ) if I [ u' v' ] T = 0 (u,v ) gradient edge (u,v) (u+u,v+v ) Lecture 14-11!
12 The aperture problem Actual mo9on Lecture 14-12!
13 The aperture problem Perceived mo9on Lecture 14-13!
14 The barber pole illusion hgp://en.wikipedia.org/wiki/barberpole_illusion Lecture 14-14!
15 The barber pole illusion hgp://en.wikipedia.org/wiki/barberpole_illusion Lecture 14-15!
16 Aperture problem cont d * From Marc Pollefeys COMP Lecture 14-16!
17 Solving the ambiguity B. Lucas and T. Kanade. An itera3ve image registra3on technique with an applica3on to stereo vision. In Proceedings of the Interna6onal Joint Conference on Ar6ficial Intelligence, pp , How to get more equa3ons for a pixel? Spa9al coherence constraint: Assume the pixel s neighbors have the same (u,v) If we use a 5x5 window, that gives us 25 equa3ons per pixel Lecture 14-17!
18 Lucas- Kanade flow Overconstrained linear system: Lecture 14-18!
19 Condi3ons for solvability When is this system solvable? What if the window contains just a single straight edge? Lecture 14-19!
20 Lucas- Kanade flow Overconstrained linear system Least squares solu3on for d given by The summa3ons are over all pixels in the K x K window Lecture 14-20!
21 Condi3ons for solvability Op3mal (u, v) sa3sfies Lucas- Kanade equa3on When is This Solvable? A T A should be inver3ble A T A should not be too small due to noise eigenvalues λ 1 and λ 2 of A T A should not be too small A T A should be well- condi3oned λ 1 / λ 2 should not be too large (λ 1 = larger eigenvalue) Does this remind anything to you? Lecture 14-21!
22 M = A T A is the second moment matrix! (Harris corner detector ) Eigenvectors and eigenvalues of A T A relate to edge direc3on and magnitude The eigenvector associated with the larger eigenvalue points in the direc3on of fastest intensity change The other eigenvector is orthogonal to it Lecture 14-22!
23 Interpre3ng the eigenvalues Classifica3on of image points using eigenvalues of the second moment matrix: λ 2 Edge λ 2 >> λ 1 Corner λ 1 and λ 2 are large, λ 1 ~ λ 2 λ 1 and λ 2 are small Flat region Edge λ 1 >> λ 2 λ 1 Lecture 14-23!
24 Edge gradients very large or very small large λ 1, small λ 2 Lecture 14-24!
25 Low- texture region gradients have small magnitude small λ 1, small λ 2 Lecture 14-25!
26 High- texture region gradients are different, large magnitudes large λ 1, large λ 2 Lecture 14-26!
27 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons Lecture 14-27!
28 What are good features to track? Can measure quality of features from just a single image Hence: tracking Harris corners (or equivalent) guarantees small error sensi3vity! à Implemented in Open CV Lecture 14-28!
29 Recap Key assump3ons (Errors in Lucas- Kanade) Small mo9on: points do not move very far Brightness constancy: projec3on of the same point looks the same in every frame Spa9al coherence: points move like their neighbors Lecture 14-29!
30 Revisi3ng the small mo3on assump3on Is this mo3on small enough? Probably not it s much larger than one pixel (2 nd order terms dominate) How might we solve this problem? * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 14-30!
31 Reduce the resolu3on! * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 14-31!
32 Mul3- resolu3on Lucas Kanade Algorithm Lecture 14-32!
33 Coarse- to- fine op3cal flow es3ma3on u=1.25 pixels u=2.5 pixels u=5 pixels image 1 H u=10 pixels image 2 I Gaussian pyramid of image 1 Gaussian pyramid of image 2 Lecture 14-33!
34 Itera3ve Refinement Iterative Lukas-Kanade Algorithm 1. Estimate velocity at each pixel by solving Lucas- Kanade equations 2. Warp I(t-1) towards I(t) using the estimated flow field - use image warping techniques 3. Repeat until convergence * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 14-34!
35 Coarse- to- fine op3cal flow es3ma3on run itera3ve L- K run itera3ve L- K warp & upsample... image 1 J Gaussian pyramid of image 1 (t) image 2 I Gaussian pyramid of image 2 (t+1) Lecture 14-35!
36 Op3cal Flow Results * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 14-36!
37 Op3cal Flow Results hgp:// OpenCV * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 14-37!
38 Recap Key assump3ons (Errors in Lucas- Kanade) Small mo9on: points do not move very far Brightness constancy: projec3on of the same point looks the same in every frame Spa9al coherence: points move like their neighbors Lecture 14-38!
39 Reminder: Gestalt common fate Lecture 14-39!
40 Mo3on segmenta3on How do we represent the mo3on in this scene? Lecture 14-40!
41 Mo3on segmenta3on J. Wang and E. Adelson. Layered Representa3on for Mo3on Analysis. CVPR Break image sequence into layers each of which has a coherent (affine) mo3on Lecture 14-41!
42 What are layers? Each layer is defined by an alpha mask and an affine mo3on model J. Wang and E. Adelson. Layered Representa3on for Mo3on Analysis. CVPR Lecture 14-42!
43 Lecture 14 -! Subs3tu3ng into the brightness constancy equa3on: y a x a a y x v y a x a a y x u ), ( ), ( + + = + + = t y x I v I u I Affine mo3on 43
44 Lecture 14 -! 0 ) ( ) ( t y x I y a x a a I y a x a a I Subs3tu3ng into the brightness constancy equa3on: y a x a a y x v y a x a a y x u ), ( ), ( + + = + + = Each pixel provides 1 linear constraint in 6 unknowns [ ] = t y x I y a x a a I y a x a a I a Err ) ( ) ( ) ( Least squares minimiza3on: Affine mo3on 44
45 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene Lecture 14-45!
46 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene 2. Iterate un3l convergence: Assign each pixel to best hypothesis Pixels with high residual error remain unassigned Lecture 14-46!
47 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene Lecture 14-47!
48 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene 2. Iterate un3l convergence: Assign each pixel to best hypothesis Pixels with high residual error remain unassigned Perform region filtering to enforce spa3al constraints Re- es3mate affine mo3ons in each region Lecture 14-48!
49 Example result J. Wang and E. Adelson. Layered Representation for Motion Analysis. CVPR Lecture 14-49!
50 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons Lecture 14-50!
51 Uses of mo3on Tracking features Segmen3ng objects based on mo3on cues Learning dynamical models Improving video quality Mo3on stabiliza3on Super resolu3on Tracking objects Recognizing events and ac3vi3es Lecture 14-51!
52 Es3ma3ng 3D structure Lecture 14-52!
53 Segmen3ng objects based on mo3on cues Background subtrac3on A sta3c camera is observing a scene Goal: separate the sta3c background from the moving foreground Lecture 14-53!
54 Segmen3ng objects based on mo3on cues Mo3on segmenta3on Segment the video into mul3ple coherently moving objects S. J. Pundlik and S. T. Birchfield, Mo3on Segmenta3on at Any Speed, Proceedings of the Bri3sh Machine Vision Conference (BMVC) 2006 Lecture 14-54!
55 Tracking objects Z.Yin and R.Collins, "On- the- fly Object Modeling while Tracking," IEEE Computer Vision and PaIern Recogni6on (CVPR '07), Minneapolis, MN, June Lecture 14-55!
56 Synthesizing dynamic textures Lecture 14-56!
57 Super- resolu3on Example: A set of low quality images Lecture 14-57!
58 Super- resolu3on Each of these images looks like this: Lecture 14-58!
59 Super- resolu3on The recovery result: Lecture 14-59!
60 Recognizing events and ac3vi3es Tracker D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI Lecture 14-60!
61 Recognizing events and ac3vi3es Juan Carlos Niebles, Hongcheng Wang and Li Fei- Fei, Unsupervised Learning of Human Ac9on Categories Using Spa9al- Temporal Words, (BMVC), Edinburgh, Lecture 14-61!
62 Recognizing events and ac3vi3es Crossing Talking Queuing Dancing jogging W. Choi & K. Shahid & S. Savarese WMC 2010 Lecture 14-62!
63 W. Choi, K. Shahid, S. Savarese, "What are they doing? : Collec3ve Ac3vity Classifica3on Using Spa3o- Temporal Rela3onship Among People", 9th Interna3onal Workshop on Visual Surveillance (VSWS09) in conjuc3on with ICCV 09 Lecture 14-63!
64 Op3cal flow without mo3on! Lecture 14-64!
65 What we have learned today? Introduc3on Op3cal flow Feature tracking Applica3ons Lecture 14-65!
Optical Flow, Motion Segmentation, Feature Tracking
BIL 719 - Computer Vision May 21, 2014 Optical Flow, Motion Segmentation, Feature Tracking Aykut Erdem Dept. of Computer Engineering Hacettepe University Motion Optical Flow Motion Segmentation Feature
Optical flow. Subhransu Maji. CMPSCI 670: Computer Vision. October 20, 2016
Optical flow Subhransu Maji CMPSC 670: Computer Vision October 20, 2016 Visual motion Man slides adapted from S. Seitz, R. Szeliski, M. Pollefes CMPSC 670 2 Motion and perceptual organization Sometimes,
Lecture 17: Face Recogni2on
Lecture 17: Face Recogni2on Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei-Fei Li Stanford Vision Lab Lecture 17-1! What we will learn today Introduc2on to face recogni2on Principal Component Analysis
Lecture 17: Face Recogni2on
Lecture 17: Face Recogni2on Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei-Fei Li Stanford Vision Lab Lecture 17-1! What we will learn today Introduc2on to face recogni2on Principal Component Analysis
Elaborazione delle Immagini Informazione multimediale - Immagini. Raffaella Lanzarotti
Elaborazione delle Immagini Informazione multimediale - Immagini Raffaella Lanzarotti OPTICAL FLOW Thanks to prof. Mubarak Shah,UCF 2 Video Video: sequence of frames (images) catch in the time Data: function
6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018
6.869 Advances in Computer Vision Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 1 Sampling Sampling Pixels Continuous world 3 Sampling 4 Sampling 5 Continuous image f (x, y) Sampling
Introduction to motion correspondence
Introduction to motion correspondence 1 IPAM - UCLA July 24, 2013 Iasonas Kokkinos Center for Visual Computing Ecole Centrale Paris / INRIA Saclay Why estimate visual motion? 2 Tracking Segmentation Structure
CS4495/6495 Introduction to Computer Vision. 6B-L1 Dense flow: Brightness constraint
CS4495/6495 Introduction to Computer Vision 6B-L1 Dense flow: Brightness constraint Motion estimation techniques Feature-based methods Direct, dense methods Motion estimation techniques Direct, dense methods
CS4670: Computer Vision Kavita Bala. Lecture 7: Harris Corner Detec=on
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detec=on Announcements HW 1 will be out soon Sign up for demo slots for PA 1 Remember that both partners have to be there We will ask you to
Motion Estimation (I) Ce Liu Microsoft Research New England
Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
Video and Motion Analysis Computer Vision Carnegie Mellon University (Kris Kitani)
Video and Motion Analysis 16-385 Computer Vision Carnegie Mellon University (Kris Kitani) Optical flow used for feature tracking on a drone Interpolated optical flow used for super slow-mo optical flow
Image Processing 1 (IP1) Bildverarbeitung 1
MIN-Fakultät Fachbereich Informatik Arbeitsbereich SAV/BV (KOGS) Image Processing 1 (IP1) Bildverarbeitung 1 Lecture 18 Mo
Image Alignment and Mosaicing
Image Alignment and Mosaicing Image Alignment Applications Local alignment: Tracking Stereo Global alignment: Camera jitter elimination Image enhancement Panoramic mosaicing Image Enhancement Original
Lecture 8: Interest Point Detection. Saad J Bedros
#1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Last Lecture : Edge Detection Preprocessing of image is desired to eliminate or at least minimize noise effects There is always tradeoff
CSCI 360 Introduc/on to Ar/ficial Intelligence Week 2: Problem Solving and Op/miza/on
CSCI 360 Introduc/on to Ar/ficial Intelligence Week 2: Problem Solving and Op/miza/on Professor Wei-Min Shen Week 13.1 and 13.2 1 Status Check Extra credits? Announcement Evalua/on process will start soon
Motion estimation. Digital Visual Effects Yung-Yu Chuang. with slides by Michael Black and P. Anandan
Motion estimation Digital Visual Effects Yung-Yu Chuang with slides b Michael Black and P. Anandan Motion estimation Parametric motion image alignment Tracking Optical flow Parametric motion direct method
Lucas-Kanade Optical Flow. Computer Vision Carnegie Mellon University (Kris Kitani)
Lucas-Kanade Optical Flow Computer Vision 16-385 Carnegie Mellon University (Kris Kitani) I x u + I y v + I t =0 I x = @I @x I y = @I u = dx v = dy I @y t = @I dt dt @t spatial derivative optical flow
Corners, Blobs & Descriptors. With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros
Corners, Blobs & Descriptors With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros Motivation: Build a Panorama M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 How do we build panorama?
Feature extraction: Corners and blobs
Feature extraction: Corners and blobs Review: Linear filtering and edge detection Name two different kinds of image noise Name a non-linear smoothing filter What advantages does median filtering have over
Computer Vision. Pa0ern Recogni4on Concepts Part I. Luis F. Teixeira MAP- i 2012/13
Computer Vision Pa0ern Recogni4on Concepts Part I Luis F. Teixeira MAP- i 2012/13 What is it? Pa0ern Recogni4on Many defini4ons in the literature The assignment of a physical object or event to one of
CS 3710: Visual Recognition Describing Images with Features. Adriana Kovashka Department of Computer Science January 8, 2015
CS 3710: Visual Recognition Describing Images with Features Adriana Kovashka Department of Computer Science January 8, 2015 Plan for Today Presentation assignments + schedule changes Image filtering Feature
Edges and Scale. Image Features. Detecting edges. Origin of Edges. Solution: smooth first. Effects of noise
Edges and Scale Image Features From Sandlot Science Slides revised from S. Seitz, R. Szeliski, S. Lazebnik, etc. Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity
Lecture 7: Finding Features (part 2/2)
Lecture 7: Finding Features (part 2/2) Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? Local invariant features MoPvaPon Requirements, invariances
Motion Estimation (I)
Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
Least Square Es?ma?on, Filtering, and Predic?on: ECE 5/639 Sta?s?cal Signal Processing II: Linear Es?ma?on
Least Square Es?ma?on, Filtering, and Predic?on: Sta?s?cal Signal Processing II: Linear Es?ma?on Eric Wan, Ph.D. Fall 2015 1 Mo?va?ons If the second-order sta?s?cs are known, the op?mum es?mator is given
Latent Dirichlet Alloca/on
Latent Dirichlet Alloca/on Blei, Ng and Jordan ( 2002 ) Presented by Deepak Santhanam What is Latent Dirichlet Alloca/on? Genera/ve Model for collec/ons of discrete data Data generated by parameters which
Least Mean Squares Regression. Machine Learning Fall 2017
Least Mean Squares Regression Machine Learning Fall 2017 1 Lecture Overview Linear classifiers What func?ons do linear classifiers express? Least Squares Method for Regression 2 Where are we? Linear classifiers
6.869 Advances in Computer Vision. Prof. Bill Freeman March 1, 2005
6.869 Advances in Computer Vision Prof. Bill Freeman March 1 2005 1 2 Local Features Matching points across images important for: object identification instance recognition object class recognition pose
Dr. Ulas Bagci
CAP5415-Computer Vision Lecture 18-Face Recogni;on Dr. Ulas Bagci bagci@ucf.edu 1 Lecture 18 Face Detec;on and Recogni;on Detec;on Recogni;on Sally 2 Why wasn t Massachusetss Bomber iden6fied by the Massachuse8s
Lecture 6: Edge Detection. CAP 5415: Computer Vision Fall 2008
Lecture 6: Edge Detection CAP 5415: Computer Vision Fall 2008 Announcements PS 2 is available Please read it by Thursday During Thursday lecture, I will be going over it in some detail Monday - Computer
Lecture 8: Interest Point Detection. Saad J Bedros
#1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Review of Edge Detectors #2 Today s Lecture Interest Points Detection What do we mean with Interest Point Detection in an Image Goal:
Lecture: Face Recognition
Lecture: Face Recognition Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 12-1 What we will learn today Introduction to face recognition The Eigenfaces Algorithm Linear
Iterative Image Registration: Lucas & Kanade Revisited. Kentaro Toyama Vision Technology Group Microsoft Research
Iterative Image Registration: Lucas & Kanade Revisited Kentaro Toyama Vision Technology Group Microsoft Research Every writer creates his own precursors. His work modifies our conception of the past, as
Unsupervised Learning: K- Means & PCA
Unsupervised Learning: K- Means & PCA Unsupervised Learning Supervised learning used labeled data pairs (x, y) to learn a func>on f : X Y But, what if we don t have labels? No labels = unsupervised learning
Robust Motion Segmentation by Spectral Clustering
Robust Motion Segmentation by Spectral Clustering Hongbin Wang and Phil F. Culverhouse Centre for Robotics Intelligent Systems University of Plymouth Plymouth, PL4 8AA, UK {hongbin.wang, P.Culverhouse}@plymouth.ac.uk
INTEREST POINTS AT DIFFERENT SCALES
INTEREST POINTS AT DIFFERENT SCALES Thank you for the slides. They come mostly from the following sources. Dan Huttenlocher Cornell U David Lowe U. of British Columbia Martial Hebert CMU Intuitively, junctions
CSE 473: Ar+ficial Intelligence
CSE 473: Ar+ficial Intelligence Hidden Markov Models Luke Ze@lemoyer - University of Washington [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188
Computer Vision Motion
Computer Vision Motion Professor Hager http://www.cs.jhu.edu/~hager 12/1/12 CS 461, Copyright G.D. Hager Outline From Stereo to Motion The motion field and optical flow (2D motion) Factorization methods
Feature detectors and descriptors. Fei-Fei Li
Feature detectors and descriptors Fei-Fei Li Feature Detection e.g. DoG detected points (~300) coordinates, neighbourhoods Feature Description e.g. SIFT local descriptors (invariant) vectors database of
Image Alignment and Mosaicing Feature Tracking and the Kalman Filter
Image Alignment and Mosaicing Feature Tracking and the Kalman Filter Image Alignment Applications Local alignment: Tracking Stereo Global alignment: Camera jitter elimination Image enhancement Panoramic
Lecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the
Lecture 7: Finding Features (part 2/2)
Lecture 7: Finding Features (part 2/2) Professor Fei- Fei Li Stanford Vision Lab Lecture 7 -! 1 What we will learn today? Local invariant features MoHvaHon Requirements, invariances Keypoint localizahon
Feature detectors and descriptors. Fei-Fei Li
Feature detectors and descriptors Fei-Fei Li Feature Detection e.g. DoG detected points (~300) coordinates, neighbourhoods Feature Description e.g. SIFT local descriptors (invariant) vectors database of
Scale & Affine Invariant Interest Point Detectors
Scale & Affine Invariant Interest Point Detectors Krystian Mikolajczyk and Cordelia Schmid Presented by Hunter Brown & Gaurav Pandey, February 19, 2009 Roadmap: Motivation Scale Invariant Detector Affine
Advances in Computer Vision. Prof. Bill Freeman. Image and shape descriptors. Readings: Mikolajczyk and Schmid; Belongie et al.
6.869 Advances in Computer Vision Prof. Bill Freeman March 3, 2005 Image and shape descriptors Affine invariant features Comparison of feature descriptors Shape context Readings: Mikolajczyk and Schmid;
Last Lecture Recap UVA CS / Introduc8on to Machine Learning and Data Mining. Lecture 3: Linear Regression
UVA CS 4501-001 / 6501 007 Introduc8on to Machine Learning and Data Mining Lecture 3: Linear Regression Yanjun Qi / Jane University of Virginia Department of Computer Science 1 Last Lecture Recap q Data
CS376 Computer Vision Lecture 6: Optical Flow
CS376 Compuer Vision Lecure 6: Opical Flow Qiing Huang Feb. 11 h 2019 Slides Credi: Krisen Grauman and Sebasian Thrun, Michael Black, Marc Pollefeys Opical Flow mage racking 3D compuaion mage sequence
Blobs & Scale Invariance
Blobs & Scale Invariance Prof. Didier Stricker Doz. Gabriele Bleser Computer Vision: Object and People Tracking With slides from Bebis, S. Lazebnik & S. Seitz, D. Lowe, A. Efros 1 Apertizer: some videos
Lecture 5. Epipolar Geometry. Professor Silvio Savarese Computational Vision and Geometry Lab. 21-Jan-15. Lecture 5 - Silvio Savarese
Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-21-Jan-15 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential
Recap: edge detection. Source: D. Lowe, L. Fei-Fei
Recap: edge detection Source: D. Lowe, L. Fei-Fei Canny edge detector 1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient 3. Non-maximum suppression: Thin multi-pixel
Methods in Computer Vision: Introduction to Optical Flow
Methods in Computer Vision: Introduction to Optical Flow Oren Freifeld Computer Science, Ben-Gurion University March 22 and March 26, 2017 Mar 22, 2017 1 / 81 A Preliminary Discussion Example and Flow
Computer Vision I. Announcements
Announcements Motion II No class Wednesda (Happ Thanksgiving) HW4 will be due Frida 1/8 Comment on Non-maximal supression CSE5A Lecture 15 Shi-Tomasi Corner Detector Filter image with a Gaussian. Compute
Global Behaviour Inference using Probabilistic Latent Semantic Analysis
Global Behaviour Inference using Probabilistic Latent Semantic Analysis Jian Li, Shaogang Gong, Tao Xiang Department of Computer Science Queen Mary College, University of London, London, E1 4NS, UK {jianli,
Lecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 11-1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed
Subsampling and image pyramids
Subsampling and image pyramids http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 3 Course announcements Homework 0 and homework 1 will be posted tonight. - Homework 0 is not required
Bellman s Curse of Dimensionality
Bellman s Curse of Dimensionality n- dimensional state space Number of states grows exponen
CS 6140: Machine Learning Spring What We Learned Last Week 2/26/16
Logis@cs CS 6140: Machine Learning Spring 2016 Instructor: Lu Wang College of Computer and Informa@on Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang Email: luwang@ccs.neu.edu Sign
Lecture 6: Finding Features (part 1/2)
Lecture 6: Finding Features (part 1/2) Professor Fei- Fei Li Stanford Vision Lab Lecture 6 -! 1 What we will learn today? Local invariant features MoHvaHon Requirements, invariances Keypoint localizahon
Camera calibration. Outline. Pinhole camera. Camera projection models. Nonlinear least square methods A camera calibration tool
Outline Camera calibration Camera projection models Camera calibration i Nonlinear least square methods A camera calibration tool Applications Digital Visual Effects Yung-Yu Chuang with slides b Richard
CS 664 Segmentation (2) Daniel Huttenlocher
CS 664 Segmentation (2) Daniel Huttenlocher Recap Last time covered perceptual organization more broadly, focused in on pixel-wise segmentation Covered local graph-based methods such as MST and Felzenszwalb-Huttenlocher
Human Pose Tracking I: Basics. David Fleet University of Toronto
Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated
Supervised object recognition, unsupervised object recognition then Perceptual organization
Supervised object recognition, unsupervised object recognition then Perceptual organization Bill Freeman, MIT 6.869 April 12, 2005 Readings Brief overview of classifiers in context of gender recognition:
Image Processing 1 (IP1) Bildverarbeitung 1
MIN-Fakultät Fachbereich Informatik Arbeitsbereich SAV/BV (KOGS Image Processing 1 (IP1 Bildverarbeitung 1 Lecture 5 Perspec
Outline. What is Machine Learning? Why Machine Learning? 9/29/08. Machine Learning Approaches to Biological Research: Bioimage Informa>cs and Beyond
Outline Machine Learning Approaches to Biological Research: Bioimage Informa>cs and Beyond Robert F. Murphy External Senior Fellow, Freiburg Ins>tute for Advanced Studies Ray and Stephanie Lane Professor
Machine vision. Summary # 4. The mask for Laplacian is given
1 Machine vision Summary # 4 The mask for Laplacian is given L = 0 1 0 1 4 1 (6) 0 1 0 Another Laplacian mask that gives more importance to the center element is L = 1 1 1 1 8 1 (7) 1 1 1 Note that the
Lecture 12. Local Feature Detection. Matching with Invariant Features. Why extract features? Why extract features? Why extract features?
Lecture 1 Why extract eatures? Motivation: panorama stitching We have two images how do we combine them? Local Feature Detection Guest lecturer: Alex Berg Reading: Harris and Stephens David Lowe IJCV We
Edge Detection. Introduction to Computer Vision. Useful Mathematics Funcs. The bad news
Edge Detection Introduction to Computer Vision CS / ECE 8B Thursday, April, 004 Edge detection (HO #5) Edge detection is a local area operator that seeks to find significant, meaningful changes in image
CSE 473/573 Computer Vision and Image Processing (CVIP)
CSE 473/573 Computer Vision and Image Processing (CVIP) Ifeoma Nwogu inwogu@buffalo.edu Lecture 11 Local Features 1 Schedule Last class We started local features Today More on local features Readings for
2D Image Processing (Extended) Kalman and particle filter
2D Image Processing (Extended) Kalman and particle filter Prof. Didier Stricker Dr. Gabriele Bleser Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz
Reduced Models for Process Simula2on and Op2miza2on
Reduced Models for Process Simulaon and Opmizaon Yidong Lang, Lorenz T. Biegler and David Miller ESI annual meeng March, 0 Models are mapping Equaon set or Module simulators Input space Reduced model Surrogate
Machine vision, spring 2018 Summary 4
Machine vision Summary # 4 The mask for Laplacian is given L = 4 (6) Another Laplacian mask that gives more importance to the center element is given by L = 8 (7) Note that the sum of the elements in the
Least Squares Parameter Es.ma.on
Least Squares Parameter Es.ma.on Alun L. Lloyd Department of Mathema.cs Biomathema.cs Graduate Program North Carolina State University Aims of this Lecture 1. Model fifng using least squares 2. Quan.fica.on
COMP 562: Introduction to Machine Learning
COMP 562: Introduction to Machine Learning Lecture 20 : Support Vector Machines, Kernels Mahmoud Mostapha 1 Department of Computer Science University of North Carolina at Chapel Hill mahmoudm@cs.unc.edu
Unit 2. Projec.ons and Subspaces
Unit. Projec.ons and Subspaces Linear Algebra and Op.miza.on MSc Bioinforma.cs for Health Sciences Eduardo Eyras Pompeu Fabra University 8-9 hkp://comprna.upf.edu/courses/master_mat/ Inner product (scalar
CSE 473: Ar+ficial Intelligence. Probability Recap. Markov Models - II. Condi+onal probability. Product rule. Chain rule.
CSE 473: Ar+ficial Intelligence Markov Models - II Daniel S. Weld - - - University of Washington [Most slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188
Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems
1 Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems V. Estivill-Castro 2 Perception Concepts Vision Chapter 4 (textbook) Sections 4.3 to 4.5 What is the course
EE 6882 Visual Search Engine
EE 6882 Visual Search Engine Prof. Shih Fu Chang, Feb. 13 th 2012 Lecture #4 Local Feature Matching Bag of Word image representation: coding and pooling (Many slides from A. Efors, W. Freeman, C. Kambhamettu,
Corner detection: the basic idea
Corner detection: the basic idea At a corner, shifting a window in any direction should give a large change in intensity flat region: no change in all directions edge : no change along the edge direction
Theory and Applica0on of Gas Turbine Systems
Theory and Applica0on of Gas Turbine Systems Part IV: Axial and Radial Flow Turbines Munich Summer School at University of Applied Sciences Prof. Kim A. Shollenberger Introduc0on to Turbines Two basic
Image Analysis. Feature extraction: corners and blobs
Image Analysis Feature extraction: corners and blobs Christophoros Nikou cnikou@cs.uoi.gr Images taken from: Computer Vision course by Svetlana Lazebnik, University of North Carolina at Chapel Hill (http://www.cs.unc.edu/~lazebnik/spring10/).
Feature Tracking. 2/27/12 ECEn 631
Corner Extraction Feature Tracking Mostly for multi-frame applications Object Tracking Motion detection Image matching Image mosaicing 3D modeling Object recognition Homography estimation... Global Features
Machine Learning and Data Mining. Linear regression. Prof. Alexander Ihler
+ Machine Learning and Data Mining Linear regression Prof. Alexander Ihler Supervised learning Notation Features x Targets y Predictions ŷ Parameters θ Learning algorithm Program ( Learner ) Change µ Improve
Lecture 3: Linear Filters
Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT)
Lecture 3: Linear Filters
Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT)
Lecture 13 Visual recognition
Lecture 13 Visual recognition Announcements Silvio Savarese Lecture 13-20-Feb-14 Lecture 13 Visual recognition Object classification bag of words models Discriminative methods Generative methods Object
Multi-Frame Factorization Techniques
Multi-Frame Factorization Techniques Suppose { x j,n } J,N j=1,n=1 is a set of corresponding image coordinates, where the index n = 1,...,N refers to the n th scene point and j = 1,..., J refers to the
Probabilistic Models of the Brain: Perception and Neural Function
Probabilistic Models of the Brain: Perception and Neural Function Edited by Rajesh P. N. Rao Bruno A. Olshausen Michael S. Lewicki The MIT Press Cambridge, Massachusetts London, England c 1 Massachusetts
UVA CS 4501: Machine Learning. Lecture 6: Linear Regression Model with Dr. Yanjun Qi. University of Virginia
UVA CS 4501: Machine Learning Lecture 6: Linear Regression Model with Regulariza@ons Dr. Yanjun Qi University of Virginia Department of Computer Science Where are we? è Five major sec@ons of this course
Sta$s$cal sequence recogni$on
Sta$s$cal sequence recogni$on Determinis$c sequence recogni$on Last $me, temporal integra$on of local distances via DP Integrates local matches over $me Normalizes $me varia$ons For cts speech, segments
STAD68: Machine Learning
STAD68: Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! h0p://www.cs.toronto.edu/~rsalakhu/ Lecture 1 Evalua;on 3 Assignments worth 40%. Midterm worth 20%. Final
Lecture 7 Rolling Constraints
Lecture 7 Rolling Constraints The most common, and most important nonholonomic constraints They cannot be wri5en in terms of the variables alone you must include some deriva9ves The resul9ng differen9al
Statistical Modeling of Visual Patterns: Part I --- From Statistical Physics to Image Models
Statistical Modeling of Visual Patterns: Part I --- From Statistical Physics to Image Models Song-Chun Zhu Departments of Statistics and Computer Science University of California, Los Angeles Natural images
Templates, Image Pyramids, and Filter Banks
Templates, Image Pyramids, and Filter Banks 09/9/ Computer Vision James Hays, Brown Slides: Hoiem and others Review. Match the spatial domain image to the Fourier magnitude image 2 3 4 5 B A C D E Slide:
Fields, wave and electromagne3c pulses. fields, waves <- > par0cles pulse <- > bunch (finite in 0me),
Fields, wave and electromagne3c pulses fields, waves par0cles pulse bunch (finite in 0me), 1 Op3cs ray or geometric op0cs: ABCD matrix, wave op0cs (used e.m. field to describe the op0cal field):
Tracking Human Heads Based on Interaction between Hypotheses with Certainty
Proc. of The 13th Scandinavian Conference on Image Analysis (SCIA2003), (J. Bigun and T. Gustavsson eds.: Image Analysis, LNCS Vol. 2749, Springer), pp. 617 624, 2003. Tracking Human Heads Based on Interaction
Feature detection.
Feature detection Kim Steenstrup Pedersen kimstp@itu.dk The IT University of Copenhagen Feature detection, The IT University of Copenhagen p.1/20 What is a feature? Features can be thought of as symbolic
Applied Time Series Analysis FISH 507. Eric Ward Mark Scheuerell Eli Holmes
Applied Time Series Analysis FISH 507 Eric Ward Mark Scheuerell Eli Holmes Introduc;ons Who are we? Who & why you re here? What are you looking to get from this class? Days and Times Lectures When: Tues
VIDEO SYNCHRONIZATION VIA SPACE-TIME INTEREST POINT DISTRIBUTION. Jingyu Yan and Marc Pollefeys
VIDEO SYNCHRONIZATION VIA SPACE-TIME INTEREST POINT DISTRIBUTION Jingyu Yan and Marc Pollefeys {yan,marc}@cs.unc.edu The University of North Carolina at Chapel Hill Department of Computer Science Chapel