Lecture 3: Linear Filters

Size: px
Start display at page:

Download "Lecture 3: Linear Filters"

Transcription

1 Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1

2 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing Some background reading: Forsyth and Ponce, Computer Vision, Chapter 7 & 8 Jae S. Lim, Two dimensional signal and image processing, Chapter 1, 4, 5 2

3 Images as functions An Image as a function f from R 2 to R M : f( x, y ) gives the intensity at position ( x, y ) Defined over a rectangle, with a finite range: f: [a,b] x [c,d ] [0,255] Domain support range f(x,y) y f(x,y) y x x 3

4 Images as functions An Image as a function f from R 2 to R M : f( x, y ) gives the intensity at position ( x, y ) Defined over a rectangle, with a finite range: f: [a,b] x [c,d ] [0,255] Domain support range A color image: rxy (, ) f( x, y) g( x, y) bxy (, ) 4

5 Images as discrete functions Images are usually digital (discrete): Sample the 2D space on a regular grid Represented as a matrix of integer values pixel 5

6 Images as discrete functions Cartesian coordinates Notation for discrete functions 6

7 Images as discrete functions Array coordinates Matlab notation 7

8 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing Some background reading: Forsyth and Ponce, Computer Vision, Chapter 7 & 8 Jae S. Lim, Two dimensional signal and image processing, Chapter 1, 4, 5 8

9 Filtering: Systems and Filters Form a new image whose pixels are a combination original pixel values Goals: Extract useful information from the images Features (edges, corners, blobs ) Modify or enhance image properties: super resolution; in painting; de noising 9

10 De noising Super resolution Salt and pepper noise In painting Bertamio et al 10

11 2D discrete space systems (filters) 11

12 Filters: Examples 2D DS moving average over a 3 3 window of neighborhood ], [ ], [ 9 1 ], )[ (, l n k m h l k f n m h f l k h

13 This image cannot currently be displayed. Moving average Courtesy of S. Seitz 13

14 Moving average (f g)[m,n] k,l f[k, l] g[m k,n 14 l] 14

15 Moving average (f g)[m,n] k,l f[k, l] g[m k,n l] 15

16 Moving average (f g)[m,n] k,l f[k, l] g[m k,n l] 16

17 Moving average (f g)[m,n] k,l f[k, l] g[m k,n l] 17

18 Moving average (f g)[m,n] k,l f[k, l] g[m k,n l] Source: S. Seitz 18

19 Moving average In summary: g[, ] Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) 19

20 Moving average 20

21 Filters: Examples Image segmentation based on a simple threshold: 21

22 Classification of systems Amplitude properties Linearity Stability Invertibility Spatial properties Causality Separability Memory Shift invariance Rotation invariance 22

23 Shift invariance If then for every input image f[n,m] and shifts n 0,m 0 Is the moving average shift invariant a system? 23

24 Is the moving average system is shift invariant?

25 Is the moving average system is shift invariant? Yes! 25

26 Linear Systems (filters) Linear filtering: Form a new image whose pixels are a weighted sum of original pixel values Use the same set of weights at each point S is a linear system (function) iff it S satisfies superposition property 26

27 Linear Systems (filters) Is the moving average a linear system? Is thresholding a linear system? f1[n,m] + f2[n,m] > T f1[n,m] < T f2[n,m]<t No! 27

28 LSI (linear shift invariant) systems Impulse response 28

29 LSI (linear shift invariant) systems Example: impulse response of the 3 by 3 moving average filter: h

30 LSI (linear shift invariant) systems An LSI system is completely specified by its impulse response. sifting property of the delta function superposition Discrete convolution 30

31 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing Some background reading: Forsyth and Ponce, Computer Vision, Chapter 7 & 8 Jae S. Lim, Two dimensional signal and image processing, Chapter 1, 4, 5 31

32 Discrete convolution Fold h[n,m] about origin to form h[ k, l] Shift the folded results by n,m to form h[n k,m l] Mul ply h[n k,m l] by f[k, l] Sum over all k,l Repeat for every n,m h[k,l] h[ k, l] h[n k,m l] n 32

33 Discrete convolution Fold h[n,m] about origin to form h[ k, l] Shift the folded results by n,m to form h[n k,m l] Mul ply h[n k,m l] by f[k, l] Sum over all k,l Repeat for every n,m h[k,l] h[ k, l] h[n k,m l] n 33

34 Discrete convolution Fold h[n,m] about origin to form h[ k, l] Shift the folded results by n,m to form h[n k,m l] Mul ply h[n k,m l] by f[k, l] Sum over all k,l Repeat for every n,m h[n k,m l] f[k,l] x h[n k,m l] f[k,l] Sum (f[k,l] x h[n k,m l]) n 34

35 Convolution in 2D examples =? Original Courtesy of D Lowe 35

36 Convolution in 2D examples = Original Filtered (no change) Courtesy of D Lowe 36

37 Convolution in 2D examples =? Original Courtesy of D Lowe 37

38 Convolution in 2D examples = Original Shifted right By 1 pixel Courtesy of D Lowe 38

39 Convolution in 2D examples =? Original Courtesy of D Lowe 39

40 Convolution in 2D examples = Original Blur (with a box filter) Courtesy of D Lowe 40

41 Convolution in 2D examples 2 - =? (Note that filter sums to 1) Original details of the image + - Courtesy of D Lowe 41

42 What does blurring take away? = original smoothed (5x5) detail Let s add it back: + a = original detail sharpened 42

43 Convolution in 2D Sharpening filter 2 - = Original Sharpening filter: Accentuates differences with local average Courtesy of D Lowe 43

44 Convolution properties Commutative property: Associative property: Distributive property: The order doesn t matter! 44

45 Convolution properties Shift property: Shift invariance: 45

46 Image support and edge effect A computer will only convolve finite support signals. That is: images that are zero for n,m outside some rectangular region MATLAB s conv2 performs 2D DS convolution of finitesupport signals. * = N2 M2 (N1 + N2 1) (M1 +M2 1) N1 M1 46

47 Image support and edge effect A computer will only convolve finite support signals. What happens at the edge? f h zero padding edge value replication mirror extension more (beyond the scope of this class) > Matlab conv2 uses zero padding 47

48 Cross correlation Cross correlation of two 2D signals f[n,m] and g[n,m] Equivalent to a convolution without the flip (k, l) is called the lag 48

49 Cross correlation example MATLAB s xcorr2 Courtesy of J. Fessler 49

50 Cross correlation example Left Right scanline Norm. corr 50

51 Convolution vs. Correlation A convolution is an integral that expresses the amount of overlap of one function as it is shifted over another function. convolution is a filtering operation Correlation compares the similarity of two sets of data. Correlation computes a measure of similarity of two input signals as they are shifted by one another. The correlation result reaches a maximum at the time when the two signals match best. correlation is a measure of relatedness of two signals

52 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing Some background reading: Forsyth and Ponce, Computer Vision, Chapter 7 & 8 Jae S. Lim, Two dimensional signal and image processing, Chapter 1, 4, 5 52

53 2D Discrete Signal Fourier Transform (DSFT) 2D discrete signal Fourier transform (DSFT) of a 2D discrete space signal g[n,m]: Inverse 2D DSFT: For memory refresher: Forsyth and Ponce, Ch 8 53

54 54

55 DSFT properties Shift: Convolution: Delta function: 55

56 Example: DSFT of moving average filter 56 ], [ ], [ 9 1 ], )[ (, l n k m h l k f n m h f l k h

57

58 Why is DFT important? Perform efficient linear convolution as product of DFTs Each DFT can be implemented using the FFT (Fast Fourier Transform ) [see appendix for details] 58

59 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing Some background reading: Forsyth and Ponce, Computer Vision, Chapter 7 & 8 Jae S. Lim, Two dimensional signal and image processing, Chapter 1, 4, 5 59

60 Sampling Throw away every other row and column to create a 1/2 size image 60

61 Sampling Down sampling operation: (trivial form of image compression) 61

62 Why is a multi scale representation useful? Find template matches at all scales e.g., when finding hands or faces, we don t know what size they will be in a particular image Template size is constant, but image size changes Efficient search for correspondence look at coarse scales, then refine with finer scales Examining all levels of detail Find edges with different amounts of blur Find textures with different spatial frequencies (levels of detail) Slide credit: David Lowe (UBC) 62

63 Things can go wrong! Aliasing 63

64 Things can go wrong! Aliasing 64

65 65

66 Sampling Theorem (Nyquist) When sampling a signal at discrete intervals, the sampling frequency must be 2 f max f max = max frequency of the input signal. v v v good bad 66

67 g[n,m] Small sampling period N g sampled [n,m] y N = sampling period T = periodicity of the replicas of the DFT (g) T T 1/N x 67

68 g[n,m] Large sampling period N g sampled [n,m] y N = sampling period T = periodicity of the replicas of the DFT (g) T 1/N x 68

69 Anti aliasing Solutions: Sample more often Get rid of all frequencies that are greater than half the new sampling frequency Will lose information but it s better than aliasing Apply a smoothing filter to remove high frequencies 69

70 Anti aliasing Apply a smoothing filter to remove high frequencies: G( ) x Low pass filter (smoothing) 70

71 Sampling algorithm 71

72 Resampling with Prior Smoothing Note: We cannot recover the high frequencies, but we can avoid artifacts by smoothing before resampling. Image Source: Forsyth & Ponce

73 The Gaussian Pyramid Low resolution G 4 ( G 3 * gaussian) 2 G 3 ( G 2 * gaussian blur ) 2 blur G 2 ( G 1 * gaussian) 2 blur G 1 ( G 0 * gaussian) 2 G 0 Image blur High resolution 73

74 Gaussian Pyramid Stored Information All the extra levels add very little overhead for memory or computation! Source: Irani & Basri 74

75 Summary: Gaussian Pyramid Construction: create each level from previous one Smooth and sample Smooth with Gaussians, in part because a Gaussian*Gaussian = another Gaussian G( 1 ) * G( 2 ) = G(sqrt( )) Gaussians are low pass filters, so the representation is redundant once smoothing has been performed. There is no need to store smoothed images at the full original resolution. Slide credit: David Lowe 75

76 Application: Vision system for TV remote control - uses template matching Figure from Computer Vision for Interactive Computer Graphics, W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE 76

77 What we have learned today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT) Sampling and aliasing 77

78 Appendix 78

79 Convergence If g absolutely summable: then If g is square summable (energy signal): 79

80 80

81 Fast Fourier transforms (FFT) Brute force evaluation of the 2D DFT would require O((NM)2) flops DFT is a separable operation we can reduce greatly the computation 81

82 Fast Fourier transforms (FFT) DFT is a separable operation: Apply the 1D DFT to each column of the image, and then apply the 1D DFT to each row of the result. Use the fast Fourier transform (1 D FFT) for these 1D DFTs! 82

83 Fast Fourier transforms (FFT) FFT computational efficiency: inner set of 1D FFTs : N O(M logm) outer set of 1D FFTs: M O(N logn) Total: O(MN logmn) flops A critical property of FFT is that N = 2 k with k =integer If x = 512x512 image saving is a factor of relative to the brute force 2D DFT! Matlab: fft2 = fft(fft(x). ). 83

84 FFT & Efficiency General goal: perform efficient linear convolution Perform convolution as product of DFTs Pros: DFT can be implemented using the FFT (fast fourier transform) FFT is very efficient (fast!) Cons: DFT perform circular convolution Compensate the wrap around effect Cons: Online memory storage Use the overlap add method or overlap save method 84

85 FFT & Efficiency Suppose we wish to convolve a image with a filter. The result will be The smallest prime factors of 272 is 2. So one could pad to a image Note: only 28% of the final image would be the part we care about the rest would be zero in exact arithmetic. Handling to a image requires much memory Use overlap add method 85

Lecture 3: Linear Filters

Lecture 3: Linear Filters Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT)

More information

Computer Vision Lecture 3

Computer Vision Lecture 3 Computer Vision Lecture 3 Linear Filters 03.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Demo Haribo Classification Code available on the class website...

More information

I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University. Computer Vision: 4. Filtering

I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University. Computer Vision: 4. Filtering I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University Computer Vision: 4. Filtering Outline Impulse response and convolution. Linear filter and image pyramid. Textbook: David A. Forsyth

More information

Templates, Image Pyramids, and Filter Banks

Templates, Image Pyramids, and Filter Banks Templates, Image Pyramids, and Filter Banks 09/9/ Computer Vision James Hays, Brown Slides: Hoiem and others Review. Match the spatial domain image to the Fourier magnitude image 2 3 4 5 B A C D E Slide:

More information

CITS 4402 Computer Vision

CITS 4402 Computer Vision CITS 4402 Computer Vision Prof Ajmal Mian Adj/A/Prof Mehdi Ravanbakhsh, CEO at Mapizy (www.mapizy.com) and InFarm (www.infarm.io) Lecture 04 Greyscale Image Analysis Lecture 03 Summary Images as 2-D signals

More information

Computer Vision Lecture 3

Computer Vision Lecture 3 Demo Haribo Classification Computer Vision Lecture 3 Linear Filters 23..24 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Code available on the class website...

More information

Lecture 04 Image Filtering

Lecture 04 Image Filtering Institute of Informatics Institute of Neuroinformatics Lecture 04 Image Filtering Davide Scaramuzza 1 Lab Exercise 2 - Today afternoon Room ETH HG E 1.1 from 13:15 to 15:00 Work description: your first

More information

Edge Detection. Introduction to Computer Vision. Useful Mathematics Funcs. The bad news

Edge Detection. Introduction to Computer Vision. Useful Mathematics Funcs. The bad news Edge Detection Introduction to Computer Vision CS / ECE 8B Thursday, April, 004 Edge detection (HO #5) Edge detection is a local area operator that seeks to find significant, meaningful changes in image

More information

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 8- Linear Filters From Spatial Domain to Frequency Domain Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D,

More information

Image Filtering. Slides, adapted from. Steve Seitz and Rick Szeliski, U.Washington

Image Filtering. Slides, adapted from. Steve Seitz and Rick Szeliski, U.Washington Image Filtering Slides, adapted from Steve Seitz and Rick Szeliski, U.Washington The power of blur All is Vanity by Charles Allen Gillbert (1873-1929) Harmon LD & JuleszB (1973) The recognition of faces.

More information

6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018

6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 6.869 Advances in Computer Vision Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 1 Sampling Sampling Pixels Continuous world 3 Sampling 4 Sampling 5 Continuous image f (x, y) Sampling

More information

Lecture 6: Edge Detection. CAP 5415: Computer Vision Fall 2008

Lecture 6: Edge Detection. CAP 5415: Computer Vision Fall 2008 Lecture 6: Edge Detection CAP 5415: Computer Vision Fall 2008 Announcements PS 2 is available Please read it by Thursday During Thursday lecture, I will be going over it in some detail Monday - Computer

More information

Thinking in Frequency

Thinking in Frequency 09/05/17 Thinking in Frequency Computational Photography University of Illinois Derek Hoiem Administrative Matlab/linear algebra tutorial tomorrow, planned for 6:30pm Probably 1214 DCL (will send confirmation

More information

CSE 473/573 Computer Vision and Image Processing (CVIP)

CSE 473/573 Computer Vision and Image Processing (CVIP) CSE 473/573 Computer Vision and Image Processing (CVIP) Ifeoma Nwogu inwogu@buffalo.edu Lecture 11 Local Features 1 Schedule Last class We started local features Today More on local features Readings for

More information

Introduction to Computer Vision. 2D Linear Systems

Introduction to Computer Vision. 2D Linear Systems Introduction to Computer Vision D Linear Systems Review: Linear Systems We define a system as a unit that converts an input function into an output function Independent variable System operator or Transfer

More information

Introduction to Linear Systems

Introduction to Linear Systems cfl David J Fleet, 998 Introduction to Linear Systems David Fleet For operator T, input I, and response R = T [I], T satisfies: ffl homogeniety: iff T [ai] = at[i] 8a 2 C ffl additivity: iff T [I + I 2

More information

Lecture'4:' Pixels'and'Filters Lecture 3: Pixels and Filters

Lecture'4:' Pixels'and'Filters Lecture 3: Pixels and Filters Lecture'4:' SF323: Applications in Machine Learning Pixels'and'Filters Lecture 3: Pixels and Filters Dr.'Juan'Carlos'Niebles Stanford'AI'Lab Krisada Chaiyasarn, PhD Professor'Fei>Fei Li Stanford'Vision'Lab

More information

Module 3. Convolution. Aim

Module 3. Convolution. Aim Module Convolution Digital Signal Processing. Slide 4. Aim How to perform convolution in real-time systems efficiently? Is convolution in time domain equivalent to multiplication of the transformed sequence?

More information

Introduction to Computer Vision

Introduction to Computer Vision Introduction to Computer Vision Michael J. Black Sept 2009 Lecture 8: Pyramids and image derivatives Goals Images as functions Derivatives of images Edges and gradients Laplacian pyramids Code for lecture

More information

Taking derivative by convolution

Taking derivative by convolution Taking derivative by convolution Partial derivatives with convolution For 2D function f(x,y), the partial derivative is: For discrete data, we can approximate using finite differences: To implement above

More information

ECE Digital Image Processing and Introduction to Computer Vision

ECE Digital Image Processing and Introduction to Computer Vision ECE592-064 Digital Image Processing and Introduction to Computer Vision Depart. of ECE, NC State University Instructor: Tianfu (Matt) Wu Spring 2017 Outline Recap, image degradation / restoration Template

More information

Linear Operators and Fourier Transform

Linear Operators and Fourier Transform Linear Operators and Fourier Transform DD2423 Image Analysis and Computer Vision Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 13, 2013

More information

Subsampling and image pyramids

Subsampling and image pyramids Subsampling and image pyramids http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 3 Course announcements Homework 0 and homework 1 will be posted tonight. - Homework 0 is not required

More information

Announcements. Filtering. Image Filtering. Linear Filters. Example: Smoothing by Averaging. Homework 2 is due Apr 26, 11:59 PM Reading:

Announcements. Filtering. Image Filtering. Linear Filters. Example: Smoothing by Averaging. Homework 2 is due Apr 26, 11:59 PM Reading: Announcements Filtering Homework 2 is due Apr 26, :59 PM eading: Chapter 4: Linear Filters CSE 52 Lecture 6 mage Filtering nput Output Filter (From Bill Freeman) Example: Smoothing by Averaging Linear

More information

Signal Processing COS 323

Signal Processing COS 323 Signal Processing COS 323 Digital Signals D: functions of space or time e.g., sound 2D: often functions of 2 spatial dimensions e.g. images 3D: functions of 3 spatial dimensions CAT, MRI scans or 2 space,

More information

Filtering and Edge Detection

Filtering and Edge Detection Filtering and Edge Detection Local Neighborhoods Hard to tell anything from a single pixel Example: you see a reddish pixel. Is this the object s color? Illumination? Noise? The next step in order of complexity

More information

! Circular Convolution. " Linear convolution with circular convolution. ! Discrete Fourier Transform. " Linear convolution through circular

! Circular Convolution.  Linear convolution with circular convolution. ! Discrete Fourier Transform.  Linear convolution through circular Previously ESE 531: Digital Signal Processing Lec 22: April 18, 2017 Fast Fourier Transform (con t)! Circular Convolution " Linear convolution with circular convolution! Discrete Fourier Transform " Linear

More information

Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005

Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Gradients and edges Points of sharp change in an image are interesting: change in reflectance change in object change

More information

Signal Processing and Linear Systems1 Lecture 4: Characterizing Systems

Signal Processing and Linear Systems1 Lecture 4: Characterizing Systems Signal Processing and Linear Systems Lecture : Characterizing Systems Nicholas Dwork www.stanford.edu/~ndwork Our goal will be to develop a way to learn how the system behaves. In general, this is a very

More information

Reading. 3. Image processing. Pixel movement. Image processing Y R I G Q

Reading. 3. Image processing. Pixel movement. Image processing Y R I G Q Reading Jain, Kasturi, Schunck, Machine Vision. McGraw-Hill, 1995. Sections 4.-4.4, 4.5(intro), 4.5.5, 4.5.6, 5.1-5.4. 3. Image processing 1 Image processing An image processing operation typically defines

More information

Additional Pointers. Introduction to Computer Vision. Convolution. Area operations: Linear filtering

Additional Pointers. Introduction to Computer Vision. Convolution. Area operations: Linear filtering Additional Pointers Introduction to Computer Vision CS / ECE 181B andout #4 : Available this afternoon Midterm: May 6, 2004 W #2 due tomorrow Ack: Prof. Matthew Turk for the lecture slides. See my ECE

More information

Outline. Convolution. Filtering

Outline. Convolution. Filtering Filtering Outline Convolution Filtering Logistics HW1 HW2 - out tomorrow Recall: what is a digital (grayscale) image? Matrix of integer values Images as height fields Let s think of image as zero-padded

More information

Frequency Filtering CSC 767

Frequency Filtering CSC 767 Frequency Filtering CSC 767 Outline Fourier transform and frequency domain Frequency view of filtering Hybrid images Sampling Slide: Hoiem Why does the Gaussian give a nice smooth image, but the square

More information

Why does a lower resolution image still make sense to us? What do we lose? Image:

Why does a lower resolution image still make sense to us? What do we lose? Image: 2D FREQUENCY DOMAIN The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their

More information

Edge Detection. Computer Vision P. Schrater Spring 2003

Edge Detection. Computer Vision P. Schrater Spring 2003 Edge Detection Computer Vision P. Schrater Spring 2003 Simplest Model: (Canny) Edge(x) = a U(x) + n(x) U(x)? x=0 Convolve image with U and find points with high magnitude. Choose value by comparing with

More information

Multidimensional digital signal processing

Multidimensional digital signal processing PSfrag replacements Two-dimensional discrete signals N 1 A 2-D discrete signal (also N called a sequence or array) is a function 2 defined over thex(n set 1 of, n 2 ordered ) pairs of integers: y(nx 1,

More information

ECG782: Multidimensional Digital Signal Processing

ECG782: Multidimensional Digital Signal Processing Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Filtering in the Frequency Domain http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Background

More information

Multiscale Image Transforms

Multiscale Image Transforms Multiscale Image Transforms Goal: Develop filter-based representations to decompose images into component parts, to extract features/structures of interest, and to attenuate noise. Motivation: extract

More information

CS 179: GPU Programming. Lecture 9 / Homework 3

CS 179: GPU Programming. Lecture 9 / Homework 3 CS 179: GPU Programming Lecture 9 / Homework 3 Recap Some algorithms are less obviously parallelizable : Reduction Sorts FFT (and certain recursive algorithms) Parallel FFT structure (radix-2) Bit-reversed

More information

Edges and Scale. Image Features. Detecting edges. Origin of Edges. Solution: smooth first. Effects of noise

Edges and Scale. Image Features. Detecting edges. Origin of Edges. Solution: smooth first. Effects of noise Edges and Scale Image Features From Sandlot Science Slides revised from S. Seitz, R. Szeliski, S. Lazebnik, etc. Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity

More information

Empirical Mean and Variance!

Empirical Mean and Variance! Global Image Properties! Global image properties refer to an image as a whole rather than components. Computation of global image properties is often required for image enhancement, preceding image analysis.!

More information

Computational Photography

Computational Photography Computational Photography Si Lu Spring 208 http://web.cecs.pdx.edu/~lusi/cs50/cs50_computati onal_photography.htm 04/0/208 Last Time o Digital Camera History of Camera Controlling Camera o Photography

More information

Lecture 8: Interest Point Detection. Saad J Bedros

Lecture 8: Interest Point Detection. Saad J Bedros #1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Review of Edge Detectors #2 Today s Lecture Interest Points Detection What do we mean with Interest Point Detection in an Image Goal:

More information

Basic Multi-rate Operations: Decimation and Interpolation

Basic Multi-rate Operations: Decimation and Interpolation 1 Basic Multirate Operations 2 Interconnection of Building Blocks 1.1 Decimation and Interpolation 1.2 Digital Filter Banks Basic Multi-rate Operations: Decimation and Interpolation Building blocks for

More information

Basics on 2-D 2 D Random Signal

Basics on 2-D 2 D Random Signal Basics on -D D Random Signal Spring 06 Instructor: K. J. Ray Liu ECE Department, Univ. of Maryland, College Park Overview Last Time: Fourier Analysis for -D signals Image enhancement via spatial filtering

More information

Assignment 2 Solutions

Assignment 2 Solutions EE225E/BIOE265 Spring 2012 Principles of MRI Miki Lustig Assignment 2 Solutions 1. Read Nishimura Ch. 3 2. Non-Uniform Sampling. A student has an assignment to monitor the level of Hetch-Hetchi reservoir

More information

Sampling in 1D ( ) Continuous time signal f(t) Discrete time signal. f(t) comb

Sampling in 1D ( ) Continuous time signal f(t) Discrete time signal. f(t) comb Sampling in 2D 1 Sampling in 1D Continuous time signal f(t) Discrete time signal t ( ) f [ k] = f( kt ) = f( t) δ t kt s k s f(t) comb k 2 Nyquist theorem (1D) At least 2 sample/period are needed to represent

More information

Advanced Features. Advanced Features: Topics. Jana Kosecka. Slides from: S. Thurn, D. Lowe, Forsyth and Ponce. Advanced features and feature matching

Advanced Features. Advanced Features: Topics. Jana Kosecka. Slides from: S. Thurn, D. Lowe, Forsyth and Ponce. Advanced features and feature matching Advanced Features Jana Kosecka Slides from: S. Thurn, D. Lowe, Forsyth and Ponce Advanced Features: Topics Advanced features and feature matching Template matching SIFT features Haar features 2 1 Features

More information

CS 179: GPU Programming. Lecture 9 / Homework 3

CS 179: GPU Programming. Lecture 9 / Homework 3 CS 179: GPU Programming Lecture 9 / Homework 3 Recap Some algorithms are less obviously parallelizable : Reduction Sorts FFT (and certain recursive algorithms) Parallel FFT structure (radix-2) Bit-reversed

More information

Introduction to Linear Image Processing

Introduction to Linear Image Processing Introduction to Linear Image Processing 1 IPAM - UCLA July 22, 2013 Iasonas Kokkinos Center for Visual Computing Ecole Centrale Paris / INRIA Saclay Image Sciences in a nutshell 2 Image Processing Image

More information

Images have structure at various scales

Images have structure at various scales Images have structure at various scales Frequency Frequency of a signal is how fast it changes Reflects scale of structure A combination of frequencies 0.1 X + 0.3 X + 0.5 X = Fourier transform Can we

More information

Image Enhancement in the frequency domain. GZ Chapter 4

Image Enhancement in the frequency domain. GZ Chapter 4 Image Enhancement in the frequency domain GZ Chapter 4 Contents In this lecture we will look at image enhancement in the frequency domain The Fourier series & the Fourier transform Image Processing in

More information

Digital Image Processing

Digital Image Processing Digital Image Processing Part 3: Fourier Transform and Filtering in the Frequency Domain AASS Learning Systems Lab, Dep. Teknik Room T109 (Fr, 11-1 o'clock) achim.lilienthal@oru.se Course Book Chapter

More information

Chapter 8 The Discrete Fourier Transform

Chapter 8 The Discrete Fourier Transform Chapter 8 The Discrete Fourier Transform Introduction Representation of periodic sequences: the discrete Fourier series Properties of the DFS The Fourier transform of periodic signals Sampling the Fourier

More information

Discrete-Time Systems

Discrete-Time Systems FIR Filters With this chapter we turn to systems as opposed to signals. The systems discussed in this chapter are finite impulse response (FIR) digital filters. The term digital filter arises because these

More information

Convolution. Define a mathematical operation on discrete-time signals called convolution, represented by *. Given two discrete-time signals x 1, x 2,

Convolution. Define a mathematical operation on discrete-time signals called convolution, represented by *. Given two discrete-time signals x 1, x 2, Filters Filters So far: Sound signals, connection to Fourier Series, Introduction to Fourier Series and Transforms, Introduction to the FFT Today Filters Filters: Keep part of the signal we are interested

More information

Feature extraction: Corners and blobs

Feature extraction: Corners and blobs Feature extraction: Corners and blobs Review: Linear filtering and edge detection Name two different kinds of image noise Name a non-linear smoothing filter What advantages does median filtering have over

More information

Computer Vision & Digital Image Processing

Computer Vision & Digital Image Processing Computer Vision & Digital Image Processing Image Restoration and Reconstruction I Dr. D. J. Jackson Lecture 11-1 Image restoration Restoration is an objective process that attempts to recover an image

More information

6.003: Signals and Systems. Sampling and Quantization

6.003: Signals and Systems. Sampling and Quantization 6.003: Signals and Systems Sampling and Quantization December 1, 2009 Last Time: Sampling and Reconstruction Uniform sampling (sampling interval T ): x[n] = x(nt ) t n Impulse reconstruction: x p (t) =

More information

Two-Dimensional Signal Processing and Image De-noising

Two-Dimensional Signal Processing and Image De-noising Two-Dimensional Signal Processing and Image De-noising Alec Koppel, Mark Eisen, Alejandro Ribeiro March 12, 2018 Until now, we considered (one-dimensional) discrete signals of the form x : [0, N 1] C of

More information

LTI H. the system H when xn [ ] is the input.

LTI H. the system H when xn [ ] is the input. REVIEW OF 1D LTI SYSTEMS LTI xn [ ] y[ n] H Operator notation: y[ n] = H{ x[ n] } In English, this is read: yn [ ] is the output of the system H when xn [ ] is the input. 2 THE MOST IMPORTANT PROPERTIES

More information

Fourier Transforms 1D

Fourier Transforms 1D Fourier Transforms 1D 3D Image Processing Alireza Ghane 1 Overview Recap Intuitions Function representations shift-invariant spaces linear, time-invariant (LTI) systems complex numbers Fourier Transforms

More information

3. Lecture. Fourier Transformation Sampling

3. Lecture. Fourier Transformation Sampling 3. Lecture Fourier Transformation Sampling Some slides taken from Digital Image Processing: An Algorithmic Introduction using Java, Wilhelm Burger and Mark James Burge Separability ² The 2D DFT can be

More information

ECG782: Multidimensional Digital Signal Processing

ECG782: Multidimensional Digital Signal Processing Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Spring 2014 TTh 14:30-15:45 CBC C313 Lecture 05 Image Processing Basics 13/02/04 http://www.ee.unlv.edu/~b1morris/ecg782/

More information

Digital Signal Processing Lecture 10 - Discrete Fourier Transform

Digital Signal Processing Lecture 10 - Discrete Fourier Transform Digital Signal Processing - Discrete Fourier Transform Electrical Engineering and Computer Science University of Tennessee, Knoxville November 12, 2015 Overview 1 2 3 4 Review - 1 Introduction Discrete-time

More information

Blobs & Scale Invariance

Blobs & Scale Invariance Blobs & Scale Invariance Prof. Didier Stricker Doz. Gabriele Bleser Computer Vision: Object and People Tracking With slides from Bebis, S. Lazebnik & S. Seitz, D. Lowe, A. Efros 1 Apertizer: some videos

More information

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. CEE598 - Visual Sensing for Civil nfrastructure Eng. & Mgmt. Session 9- mage Detectors, Part Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering Lab e-mail:

More information

Colorado School of Mines Image and Multidimensional Signal Processing

Colorado School of Mines Image and Multidimensional Signal Processing Image and Multidimensional Signal Processing Professor William Hoff Department of Electrical Engineering and Computer Science Spatial Filtering Main idea Spatial filtering Define a neighborhood of a pixel

More information

ENT 315 Medical Signal Processing CHAPTER 2 DISCRETE FOURIER TRANSFORM. Dr. Lim Chee Chin

ENT 315 Medical Signal Processing CHAPTER 2 DISCRETE FOURIER TRANSFORM. Dr. Lim Chee Chin ENT 315 Medical Signal Processing CHAPTER 2 DISCRETE FOURIER TRANSFORM Dr. Lim Chee Chin Outline Introduction Discrete Fourier Series Properties of Discrete Fourier Series Time domain aliasing due to frequency

More information

Multiresolution Analysis

Multiresolution Analysis Multiresolution Analysis DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Frames Short-time Fourier transform

More information

Fourier Analysis and Imaging Ronald Bracewell L.M. Terman Professor of Electrical Engineering Emeritus Stanford University Stanford, California

Fourier Analysis and Imaging Ronald Bracewell L.M. Terman Professor of Electrical Engineering Emeritus Stanford University Stanford, California Fourier Analysis and Imaging Ronald Bracewell L.M. Terman Professor of Electrical Engineering Emeritus Stanford University Stanford, California 4u Springer Contents fa 1 PREFACE INTRODUCTION xiii 1 11

More information

Neural Networks 2. 2 Receptive fields and dealing with image inputs

Neural Networks 2. 2 Receptive fields and dealing with image inputs CS 446 Machine Learning Fall 2016 Oct 04, 2016 Neural Networks 2 Professor: Dan Roth Scribe: C. Cheng, C. Cervantes Overview Convolutional Neural Networks Recurrent Neural Networks 1 Introduction There

More information

ECE 425. Image Science and Engineering

ECE 425. Image Science and Engineering ECE 425 Image Science and Engineering Spring Semester 2000 Course Notes Robert A. Schowengerdt schowengerdt@ece.arizona.edu (520) 62-2706 (voice), (520) 62-8076 (fa) ECE402 DEFINITIONS 2 Image science

More information

Filtering, Frequency, and Edges

Filtering, Frequency, and Edges CS450: Introduction to Computer Vision Filtering, Frequency, and Edges Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC),

More information

Today s lecture. Local neighbourhood processing. The convolution. Removing uncorrelated noise from an image The Fourier transform

Today s lecture. Local neighbourhood processing. The convolution. Removing uncorrelated noise from an image The Fourier transform Cris Luengo TD396 fall 4 cris@cbuuse Today s lecture Local neighbourhood processing smoothing an image sharpening an image The convolution What is it? What is it useful for? How can I compute it? Removing

More information

Blob Detection CSC 767

Blob Detection CSC 767 Blob Detection CSC 767 Blob detection Slides: S. Lazebnik Feature detection with scale selection We want to extract features with characteristic scale that is covariant with the image transformation Blob

More information

! Introduction. ! Discrete Time Signals & Systems. ! Z-Transform. ! Inverse Z-Transform. ! Sampling of Continuous Time Signals

! Introduction. ! Discrete Time Signals & Systems. ! Z-Transform. ! Inverse Z-Transform. ! Sampling of Continuous Time Signals ESE 531: Digital Signal Processing Lec 25: April 24, 2018 Review Course Content! Introduction! Discrete Time Signals & Systems! Discrete Time Fourier Transform! Z-Transform! Inverse Z-Transform! Sampling

More information

Spatial-Domain Convolution Filters

Spatial-Domain Convolution Filters Spatial-Domain Filtering 9 Spatial-Domain Convolution Filters Consider a linear space-invariant (LSI) system as shown: The two separate inputs to the LSI system, x 1 (m) and x 2 (m), and their corresponding

More information

IT DIGITAL SIGNAL PROCESSING (2013 regulation) UNIT-1 SIGNALS AND SYSTEMS PART-A

IT DIGITAL SIGNAL PROCESSING (2013 regulation) UNIT-1 SIGNALS AND SYSTEMS PART-A DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING IT6502 - DIGITAL SIGNAL PROCESSING (2013 regulation) UNIT-1 SIGNALS AND SYSTEMS PART-A 1. What is a continuous and discrete time signal? Continuous

More information

Computer Vision. Filtering in the Frequency Domain

Computer Vision. Filtering in the Frequency Domain Computer Vision Filtering in the Frequency Domain Filippo Bergamasco (filippo.bergamasco@unive.it) http://www.dais.unive.it/~bergamasco DAIS, Ca Foscari University of Venice Academic year 2016/2017 Introduction

More information

EE 225D LECTURE ON DIGITAL FILTERS. University of California Berkeley

EE 225D LECTURE ON DIGITAL FILTERS. University of California Berkeley University of California Berkeley College of Engineering Department of Electrical Engineering and Computer Sciences Professors : N.Morgan / B.Gold EE225D Digital Filters Spring,1999 Lecture 7 N.MORGAN

More information

Corners, Blobs & Descriptors. With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros

Corners, Blobs & Descriptors. With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros Corners, Blobs & Descriptors With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros Motivation: Build a Panorama M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 How do we build panorama?

More information

Lecture Notes 5: Multiresolution Analysis

Lecture Notes 5: Multiresolution Analysis Optimization-based data analysis Fall 2017 Lecture Notes 5: Multiresolution Analysis 1 Frames A frame is a generalization of an orthonormal basis. The inner products between the vectors in a frame and

More information

Theory and Problems of Signals and Systems

Theory and Problems of Signals and Systems SCHAUM'S OUTLINES OF Theory and Problems of Signals and Systems HWEI P. HSU is Professor of Electrical Engineering at Fairleigh Dickinson University. He received his B.S. from National Taiwan University

More information

Introduction to Sparsity in Signal Processing

Introduction to Sparsity in Signal Processing 1 Introduction to Sparsity in Signal Processing Ivan Selesnick Polytechnic Institute of New York University Brooklyn, New York selesi@poly.edu 212 2 Under-determined linear equations Consider a system

More information

The Frequency Domain, without tears. Many slides borrowed from Steve Seitz

The Frequency Domain, without tears. Many slides borrowed from Steve Seitz The Frequency Domain, without tears Many slides borrowed from Steve Seitz Somewhere in Cinque Terre, May 2005 CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2016

More information

Introduction to Sparsity in Signal Processing

Introduction to Sparsity in Signal Processing 1 Introduction to Sparsity in Signal Processing Ivan Selesnick Polytechnic Institute of New York University Brooklyn, New York selesi@poly.edu 212 2 Under-determined linear equations Consider a system

More information

CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS

CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS LAST TIME Intro to cudnn Deep neural nets using cublas and cudnn TODAY Building a better model for image classification Overfitting

More information

Digital Image Processing COSC 6380/4393

Digital Image Processing COSC 6380/4393 Digital Image Processing COSC 6380/4393 Lecture 13 Oct 2 nd, 2018 Pranav Mantini Slides from Dr. Shishir K Shah, and Frank Liu Review f 0 0 0 1 0 0 0 0 w 1 2 3 2 8 Zero Padding 0 0 0 0 0 0 0 1 0 0 0 0

More information

Machine vision. Summary # 4. The mask for Laplacian is given

Machine vision. Summary # 4. The mask for Laplacian is given 1 Machine vision Summary # 4 The mask for Laplacian is given L = 0 1 0 1 4 1 (6) 0 1 0 Another Laplacian mask that gives more importance to the center element is L = 1 1 1 1 8 1 (7) 1 1 1 Note that the

More information

EENG580 Game 2: Verbal Representation of Course Content

EENG580 Game 2: Verbal Representation of Course Content Intro to DSP Game 2 Verbal Representation of Course Content EENG580 Game 2: Verbal Representation of Course Content Activity summary Overview: Word game similar to Taboo, Catch Phrase, Unspeakable, Battle

More information

Sound & Vibration Magazine March, Fundamentals of the Discrete Fourier Transform

Sound & Vibration Magazine March, Fundamentals of the Discrete Fourier Transform Fundamentals of the Discrete Fourier Transform Mark H. Richardson Hewlett Packard Corporation Santa Clara, California The Fourier transform is a mathematical procedure that was discovered by a French mathematician

More information

Achieving scale covariance

Achieving scale covariance Achieving scale covariance Goal: independently detect corresponding regions in scaled versions of the same image Need scale selection mechanism for finding characteristic region size that is covariant

More information

Review Smoothing Spatial Filters Sharpening Spatial Filters. Spatial Filtering. Dr. Praveen Sankaran. Department of ECE NIT Calicut.

Review Smoothing Spatial Filters Sharpening Spatial Filters. Spatial Filtering. Dr. Praveen Sankaran. Department of ECE NIT Calicut. Spatial Filtering Dr. Praveen Sankaran Department of ECE NIT Calicut January 7, 203 Outline 2 Linear Nonlinear 3 Spatial Domain Refers to the image plane itself. Direct manipulation of image pixels. Figure:

More information

Lecture 20: Discrete Fourier Transform and FFT

Lecture 20: Discrete Fourier Transform and FFT EE518 Digital Signal Processing University of Washington Autumn 2001 Dept of Electrical Engineering Lecture 20: Discrete Fourier Transform and FFT Dec 10, 2001 Prof: J Bilmes TA:

More information

Detectors part II Descriptors

Detectors part II Descriptors EECS 442 Computer vision Detectors part II Descriptors Blob detectors Invariance Descriptors Some slides of this lectures are courtesy of prof F. Li, prof S. Lazebnik, and various other lecturers Goal:

More information

Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems

Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems 1 Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems V. Estivill-Castro 2 Perception Concepts Vision Chapter 4 (textbook) Sections 4.3 to 4.5 What is the course

More information

V(t) = Total Power = Calculating the Power Spectral Density (PSD) in IDL. Thomas Ferree, Ph.D. August 23, 1999

V(t) = Total Power = Calculating the Power Spectral Density (PSD) in IDL. Thomas Ferree, Ph.D. August 23, 1999 Calculating the Power Spectral Density (PSD) in IDL Thomas Ferree, Ph.D. August 23, 1999 This note outlines the calculation of power spectra via the fast Fourier transform (FFT) algorithm. There are several

More information

Machine vision, spring 2018 Summary 4

Machine vision, spring 2018 Summary 4 Machine vision Summary # 4 The mask for Laplacian is given L = 4 (6) Another Laplacian mask that gives more importance to the center element is given by L = 8 (7) Note that the sum of the elements in the

More information