Super-Resolution. Dr. Yossi Rubner. Many slides from Miki Elad - Technion
|
|
- Adam Simon Watkins
- 5 years ago
- Views:
Transcription
1 Super-Resolution Dr. Yossi Rubner Many slides from Mii Elad - Technion 5/5/2007
2 53 images, ratio :4 Example - Video
3 40 images ratio :4 Example Surveillance
4 Example Enhance Mosaics
5
6 Super-Resolution - Agenda The basic idea Image formation process Formulation and solution Maximum lielihood Robust solution MAP using bilateral filter Special cases and related problems Translational Motion Interpolation / Denoising / Deblurring SR in time
7 Intuition For a given band-limited image, the Nyquist sampling theorem states that if a uniform sampling is fine enough ( D), perfect reconstruction is possible. D D
8 Intuition Due to our limited camera resolution, we sample using an insufficient 2D grid 2D 2D
9 Intuition However, if we tae a second picture, shifting the camera slightly to the right we obtain: 2D 2D
10 Intuition Similarly, by shifting down we get a third image: 2D 2D
11 Intuition And finally, by shifting down and to the right we get the fourth image: 2D 2D
12 It is trivial to see that interlacing the four images, we get that the desired resolution is obtained, and thus perfect reconstruction is guaranteed. Intuition
13 What if the camera displacement is Arbitrary? What if the camera rotates? Gets closer to the object (zoom)? Rotation/Scale/Disp.
14 There is no sampling theorem covering this case Rotation/Scale/Disp.
15 Further Complications Complicated motion perspective, local motion, Blur sampling is not a point operation Different effect depending on geometrical warp Noise
16 Super-Resolution - Agenda The basic idea Image formation process Formulation and solution Maximum lielihood Robust solution MAP using bilateral filter Special cases and related problems Translational Motion Interpolation / Denoising / Deblurring SR in time
17 Image Formation Scene HR Geometric transformation F Optical Blur H Sampling D Noise LR Can we write these steps as linear operators? HR D H F LR
18 Geometric Transformation F Scene Geometric transformation Any appropriate motion model Every frame has different transformation Usually found by a separate registration algorithm
19 Geometric Transformation Can be modeled as a linear operation F F F F
20 Optical Blur H Geometric transformation Optical Blur Due to the lens PSF Usually H H
21 Optical Blur Can be modeled as a linear operation H H H H
22 Sampling D Optical Blur Sampling Pixel operation consists of area integration followed by decimation The integration can be part of H D is the decimation only Usually D D
23 H PSF * PIEL H
24 Fill Factor Real pixel area Geometrically, the fill-factor is less than 80% for circular microlenses since the area of an unit circle is π/ Square-shaped micro-lenses have poor focusing capabilities
25 Decimation Can be modeled as a linear operation D D D o 0 0 o D 0
26 Super-Resolution - Agenda The basic idea Image formation process Formulation and solution Maximum lielihood Robust solution MAP using bilateral filter Special cases and related problems Translational Motion Interpolation / Denoising / Deblurring SR in time
27 Super-Resolution - Model Geometric warp Blur Decimation High- Resolution Image F I H D V Y Low- Resolution Exposures Additive Noise F N H N D N Y N Y V N { 2 0, } D H F + V, V ~ N σ n N
28 Simplified Model Geometric warp Blur Decimation High- Resolution Image F I H D V Y Low- Resolution Exposures Additive Noise F N H D Y N Y V N { 2 0, } DHF + V, V ~ N σ n N
29 The Super-Resolution Problem Y DHF + V { 0, 2 } σ, V ~ N n Given Y The measured images (noisy, blurry, down-sampled..) H The blur can be extracted from the camera characteristics D The decimation is dictated by the required resolution ratio F The warp can be estimated using motion estimation σ n The noise can be extracted from the camera / image Recover HR image
30 The Model as One Equation 200,000 Y Y M Y 2 N DHF DHF2 M DHFN V V + M V 2 N r resolution factor 4 MM size of the frames 0000 N number of frames 20 F H D Y ofsize ofsize ofsize ofsize ofsize [ r M r M ] [ r M r M ] [ M r M ] 2 2 [ r M ] 2 [ M ] [60,00060,000] [60,00060,000] [0,00060,000] [60,000] [0,000] Linear algebra notation is intended only to develop algorithm
31 SR - Solutions Maximum Lielihood (ML): argmin N N DHF Y 2 Often ill posed problem! Maximum Aposteriori Probability (MAP) argmin DHF Y 2 { } +λa Smoothness constraint regularization
32 ML Reconstruction (LS) ( ) N ML Y 2 2 DHF ε Minimize: ( ) ( ) 0 ˆ 2 2 N T T T ML Y DHF H D F ε Thus, require: N T T T N T T T Y ˆ D H F DHF D H F A B B A ˆ
33 LS - Iterative Solution Steepest descent ˆ N T T T n+ ˆ β n F H D n ( DHF ˆ Y ) Bac projection Simulated error All the above operations can be interpreted as operations performed on images. There is no actual need to use the Matrix-Vector notations as shown here.
34 LS - Iterative Solution Steepest descent ˆ N T T T n+ ˆ n β F H D n ( DHF ˆ Y ) For..N ˆ n Y geometry wrap convolve with H down sample - up sample convolve with H T inverse geometry wrap F H D T D T H T F -β ˆ n+
35 Example HR image LR + noise 4 Least squares Simulated example from Farisu at al. IEEE trans. On Image Processing, 04
36 Robust Reconstruction Cases of measurements outlier: Some of the images are irrelevant Error in motion estimation, Error in the blur function General model mismatch
37 Robust Reconstruction Minimize: ε 2 N ( ) DHF Y ˆ N T T T n+ ˆ β n F H D sign n ( DHF ˆ Y )
38 Robust Reconstruction Steepest descent For..N ˆ n ˆ N T T T n+ ˆ n β F H D sign n Y ( DHF ˆ Y ) geometry wrap convolve with H down sample - sign up sample convolve with H T inverse geometry wrap F H D T D T H T F -β ˆ n+
39 Example - Outliers Least squares HR image LR + noise 4 Simulated example from Farisu at al. IEEE trans. On Image Processing, 04 Robust Reconstruction
40 Example Registration Error L 2 norm based L norm based 20 images, ratio :4
41 MAP Reconstruction 2 MAP N ( ) DHF Y λa{ } ε + Regularization term: 2 Tihonov cost function A T { } Γ 2 Total variation Bilateral filter A TV { } A B P P { } l P m P α l + m S l x S m y
42 Robust Estimation + Regularization ( ) + + P P l P P m m y l x m l N S S Y 2 α λ ε DHF Minimize: ( ) [ ] ( ) P P l P P m n m y l x n m y l x m l N n T T T n n S S S S I Y ˆ ˆ sign ˆ sign ˆ ˆ α λ β DHF H D F
43 Robust Estimation + Regularization ˆ β N P P l + m ( ) + [ ] ( ) l m l m DHF ˆ Y λ I S S sign ˆ S S ˆ T T T ˆ n F H D sign n n+ α l P m P x y n x y n For..N geometry wrap convolve with H down sample Y - sign up sample convolve with H T inverse geometry wrap ˆ n For l,m-p..p -β ˆ + n horizontal shift l vertical shift m - - m + l sign horizontal shift -l vertical shift -m λα From Farisu at al. IEEE trans. On Image Processing, 04
44 Example 8 frames Resolution factor of 4 From Farisu at al. IEEE trans. On Image Processing, 04
45 Images from Vigilant Ltd.
46
47 Super-Resolution - Agenda The basic idea Image formation process Formulation and solution Maximum lielihood Robust solution MAP using bilateral filter Special cases and related problems Translational Motion Interpolation / Denoising / Deblurring SR in time Limitations of Super-Resolution
48 Special Case Translational Motion In this case H and F commute: T T HF F H H F F T H T Y DHF + V DF H + V DF Z + V Z H SR is decomposed into 2 steps. Find blur HR image from LR images non-iterative 2. Deconvolve the result using H iterative
49 Intuition Y DF Z + V Z H PSF* ZPIEL*PSF* Using the samples can, at most, reconstruct Z To recover, need to deconvolve Z
50 Step I Find Blurred HR Minimize: ε 2 ML N ( Z) DF Z Y 2 L 2 For all frames, copy registered pixels to HR grid and average [Elad & Hel-Or, 0] L For all frames, copy registered pixels to HR grid and use median [Farisu, 04]
51 Solution for L 2 ( ) N ML Y Z Z 2 2 DF ε Minimize: ( ) 0 2 Z Z ε ML Thus, require: N T T N T T Y P D F DF D F R Z P ˆ R Average of HR grid Diagonal, number Of occurrences per HR grid
52 Step II - Deblur Minimize: ( ) H Z λa{ } 2 ε + ˆ ˆ β ˆ { ( ) + { } T H signhˆ Z A ˆ n+ n n λ n n
53 Example 6464 LR Before deblur After deblur From Pham at al. Proc. Of SPIE-IS&T, 05. Simulated.
54 Related Problems Denoising (multiple frames) Y + V, V { 0, σ 2 } ~ N n Denoising (single frame) Deblurring Y + V, Y H + V, { 0, σ 2 } V ~ N n Interpolation single-image super-resolution { 0, σ 2 } V ~ N n Y DH + V, { 0, σ 2 } V ~ N n
55 Super-Resolution - Agenda The basic idea Image formation process Formulation and solution Maximum lielihood Robust solution MAP using bilateral filter Special cases and related problems Translational Motion Interpolation / Denoising / Deblurring SR in time
56 Space-Time Super-Resolution Observing events faster than frame-rate Handles: Motion aliasing Motion blur Wor and slides by Michal Irani & Yaron Caspi (ECCV 02)
57 Motion Aliasing The Wagon wheel effect: Slow-motion: time time time Continuous signal Sub-sampled in time Slow motion
58 Motion Blur
59 Space-Time Super-Resolution Low-resolution images video sequences: time time time High space-time resolution sequence: time time Super-resolution in space and in time.
60 Space-Time Super-Resolution S h (x h,y h,t h ) y t T x x y Blur ernel: PSF l S l S n Exposure time t
61 Example: Motion-Aliasing Input Input 2 Input 3 Input 4 25 [frames/sec]
62 Example: Motion-Aliasing Input sequence in slow-motion (x3): Super-resolution resolution in time (x3): 75 [frames/sec] 75 [frames/sec]
63 Example: Motion-Blur (simulation) Simulated sequences of fast event Long exposure-time Low frame-rate Non-uniform distribution in time Overlay of frames One low-res sequence: Another low-res sequence: And another one...
64 Output sequence: Output trajectory: Deblurring: Input: (x5 frame-rate) Without estimating motion of the ball! Output: 3 out of 8 low-resolution input sequences (frame overlays; trajectories):
65 Example: Motion-Blur (real) Frames 4 input at sequences: collision: Video Video 2 Output frame at collision: Video 3 Video 4
Super-Resolution. Shai Avidan Tel-Aviv University
Super-Resolution Shai Avidan Tel-Aviv University Slide Credits (partial list) Ric Szelisi Steve Seitz Alyosha Efros Yacov Hel-Or Yossi Rubner Mii Elad Marc Levoy Bill Freeman Fredo Durand Sylvain Paris
More informationSimultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors
Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Sean Borman and Robert L. Stevenson Department of Electrical Engineering, University of Notre Dame Notre Dame,
More informationWhat is Image Deblurring?
What is Image Deblurring? When we use a camera, we want the recorded image to be a faithful representation of the scene that we see but every image is more or less blurry, depending on the circumstances.
More informationITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION
ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION Michael Elad The Computer Science Department The Technion Israel Institute o technology Haia 3000, Israel * SIAM Conerence on Imaging Science
More informationMotion Estimation (I)
Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
More informationTemplates, Image Pyramids, and Filter Banks
Templates, Image Pyramids, and Filter Banks 09/9/ Computer Vision James Hays, Brown Slides: Hoiem and others Review. Match the spatial domain image to the Fourier magnitude image 2 3 4 5 B A C D E Slide:
More informationAtmospheric Turbulence Effects Removal on Infrared Sequences Degraded by Local Isoplanatism
Atmospheric Turbulence Effects Removal on Infrared Sequences Degraded by Local Isoplanatism Magali Lemaitre 1, Olivier Laligant 1, Jacques Blanc-Talon 2, and Fabrice Mériaudeau 1 1 Le2i Laboratory, University
More informationMotion Estimation (I) Ce Liu Microsoft Research New England
Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
More informationSparse & Redundant Representations by Iterated-Shrinkage Algorithms
Sparse & Redundant Representations by Michael Elad * The Computer Science Department The Technion Israel Institute of technology Haifa 3000, Israel 6-30 August 007 San Diego Convention Center San Diego,
More informationSatellite image deconvolution using complex wavelet packets
Satellite image deconvolution using complex wavelet packets André Jalobeanu, Laure Blanc-Féraud, Josiane Zerubia ARIANA research group INRIA Sophia Antipolis, France CNRS / INRIA / UNSA www.inria.fr/ariana
More informationRecent Advances in SPSA at the Extremes: Adaptive Methods for Smooth Problems and Discrete Methods for Non-Smooth Problems
Recent Advances in SPSA at the Extremes: Adaptive Methods for Smooth Problems and Discrete Methods for Non-Smooth Problems SGM2014: Stochastic Gradient Methods IPAM, February 24 28, 2014 James C. Spall
More informationEE 367 / CS 448I Computational Imaging and Display Notes: Image Deconvolution (lecture 6)
EE 367 / CS 448I Computational Imaging and Display Notes: Image Deconvolution (lecture 6) Gordon Wetzstein gordon.wetzstein@stanford.edu This document serves as a supplement to the material discussed in
More informationSingle Exposure Enhancement and Reconstruction. Some slides are from: J. Kosecka, Y. Chuang, A. Efros, C. B. Owen, W. Freeman
Single Exposure Enhancement and Reconstruction Some slides are from: J. Kosecka, Y. Chuang, A. Efros, C. B. Owen, W. Freeman 1 Reconstruction as an Inverse Problem Original image f Distortion & Sampling
More information6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018
6.869 Advances in Computer Vision Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 1 Sampling Sampling Pixels Continuous world 3 Sampling 4 Sampling 5 Continuous image f (x, y) Sampling
More informationBayesian Paradigm. Maximum A Posteriori Estimation
Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)
More informationLecture 3: Linear Filters
Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT)
More informationModeling Blurred Video with Layers Supplemental material
Modeling Blurred Video with Layers Supplemental material Jonas Wulff, Michael J. Black Max Planck Institute for Intelligent Systems, Tübingen, Germany {jonas.wulff,black}@tue.mpg.de July 6, 204 Contents
More informationLecture 3: Linear Filters
Lecture 3: Linear Filters Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Images as functions Linear systems (filters) Convolution and correlation Discrete Fourier Transform (DFT)
More informationComputer Vision Lecture 3
Computer Vision Lecture 3 Linear Filters 03.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Demo Haribo Classification Code available on the class website...
More informationInverse problem and optimization
Inverse problem and optimization Laurent Condat, Nelly Pustelnik CNRS, Gipsa-lab CNRS, Laboratoire de Physique de l ENS de Lyon Decembre, 15th 2016 Inverse problem and optimization 2/36 Plan 1. Examples
More informationImage Processing in Astrophysics
AIM-CEA Saclay, France Image Processing in Astrophysics Sandrine Pires sandrine.pires@cea.fr NDPI 2011 Image Processing : Goals Image processing is used once the image acquisition is done by the telescope
More informationImage Alignment and Mosaicing Feature Tracking and the Kalman Filter
Image Alignment and Mosaicing Feature Tracking and the Kalman Filter Image Alignment Applications Local alignment: Tracking Stereo Global alignment: Camera jitter elimination Image enhancement Panoramic
More informationCovariance-Based PCA for Multi-Size Data
Covariance-Based PCA for Multi-Size Data Menghua Zhai, Feiyu Shi, Drew Duncan, and Nathan Jacobs Department of Computer Science, University of Kentucky, USA {mzh234, fsh224, drew, jacobs}@cs.uky.edu Abstract
More informationEE Camera & Image Formation
Electric Electronic Engineering Bogazici University February 21, 2018 Introduction Introduction Camera models Goal: To understand the image acquisition process. Function of the camera Similar to that of
More informationAccelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems)
Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Donghwan Kim and Jeffrey A. Fessler EECS Department, University of Michigan
More informationTRACKING SOLUTIONS OF TIME VARYING LINEAR INVERSE PROBLEMS
TRACKING SOLUTIONS OF TIME VARYING LINEAR INVERSE PROBLEMS Martin Kleinsteuber and Simon Hawe Department of Electrical Engineering and Information Technology, Technische Universität München, München, Arcistraße
More informationAtmospheric turbulence restoration by diffeomorphic image registration and blind deconvolution
Atmospheric turbulence restoration by diffeomorphic image registration and blind deconvolution Jérôme Gilles, Tristan Dagobert, Carlo De Franchis DGA/CEP - EORD department, 16bis rue Prieur de la Côte
More informationComputer Vision & Digital Image Processing
Computer Vision & Digital Image Processing Image Restoration and Reconstruction I Dr. D. J. Jackson Lecture 11-1 Image restoration Restoration is an objective process that attempts to recover an image
More informationNoise Removal? The Evolution Of Pr(x) Denoising By Energy Minimization. ( x) An Introduction to Sparse Representation and the K-SVD Algorithm
Sparse Representation and the K-SV Algorithm he CS epartment he echnion Israel Institute of technology Haifa 3, Israel University of Erlangen - Nürnberg April 8 Noise Removal? Our story begins with image
More informationSPARSE SIGNAL RESTORATION. 1. Introduction
SPARSE SIGNAL RESTORATION IVAN W. SELESNICK 1. Introduction These notes describe an approach for the restoration of degraded signals using sparsity. This approach, which has become quite popular, is useful
More informationCamera calibration. Outline. Pinhole camera. Camera projection models. Nonlinear least square methods A camera calibration tool
Outline Camera calibration Camera projection models Camera calibration i Nonlinear least square methods A camera calibration tool Applications Digital Visual Effects Yung-Yu Chuang with slides b Richard
More informationLPA-ICI Applications in Image Processing
LPA-ICI Applications in Image Processing Denoising Deblurring Derivative estimation Edge detection Inverse halftoning Denoising Consider z (x) =y (x)+η (x), wherey is noise-free image and η is noise. assume
More informationA Factorization Method for 3D Multi-body Motion Estimation and Segmentation
1 A Factorization Method for 3D Multi-body Motion Estimation and Segmentation René Vidal Department of EECS University of California Berkeley CA 94710 rvidal@eecs.berkeley.edu Stefano Soatto Dept. of Computer
More informationMotion estimation. Digital Visual Effects Yung-Yu Chuang. with slides by Michael Black and P. Anandan
Motion estimation Digital Visual Effects Yung-Yu Chuang with slides b Michael Black and P. Anandan Motion estimation Parametric motion image alignment Tracking Optical flow Parametric motion direct method
More informationUncertainty Models in Quasiconvex Optimization for Geometric Reconstruction
Uncertainty Models in Quasiconvex Optimization for Geometric Reconstruction Qifa Ke and Takeo Kanade Department of Computer Science, Carnegie Mellon University Email: ke@cmu.edu, tk@cs.cmu.edu Abstract
More informationRegularization methods for large-scale, ill-posed, linear, discrete, inverse problems
Regularization methods for large-scale, ill-posed, linear, discrete, inverse problems Silvia Gazzola Dipartimento di Matematica - Università di Padova January 10, 2012 Seminario ex-studenti 2 Silvia Gazzola
More informationENERGY METHODS IN IMAGE PROCESSING WITH EDGE ENHANCEMENT
ENERGY METHODS IN IMAGE PROCESSING WITH EDGE ENHANCEMENT PRASHANT ATHAVALE Abstract. Digital images are can be realized as L 2 (R 2 objects. Noise is introduced in a digital image due to various reasons.
More informationDictionary Learning for photo-z estimation
Dictionary Learning for photo-z estimation Joana Frontera-Pons, Florent Sureau, Jérôme Bobin 5th September 2017 - Workshop Dictionary Learning on Manifolds MOTIVATION! Goal : Measure the radial positions
More informationScience Insights: An International Journal
Available online at http://www.urpjournals.com Science Insights: An International Journal Universal Research Publications. All rights reserved ISSN 2277 3835 Original Article Object Recognition using Zernike
More informationWavelet Footprints: Theory, Algorithms, and Applications
1306 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 51, NO. 5, MAY 2003 Wavelet Footprints: Theory, Algorithms, and Applications Pier Luigi Dragotti, Member, IEEE, and Martin Vetterli, Fellow, IEEE Abstract
More informationCITS 4402 Computer Vision
CITS 4402 Computer Vision Prof Ajmal Mian Adj/A/Prof Mehdi Ravanbakhsh, CEO at Mapizy (www.mapizy.com) and InFarm (www.infarm.io) Lecture 04 Greyscale Image Analysis Lecture 03 Summary Images as 2-D signals
More informationIterative Image Registration: Lucas & Kanade Revisited. Kentaro Toyama Vision Technology Group Microsoft Research
Iterative Image Registration: Lucas & Kanade Revisited Kentaro Toyama Vision Technology Group Microsoft Research Every writer creates his own precursors. His work modifies our conception of the past, as
More informationPreconditioning. Noisy, Ill-Conditioned Linear Systems
Preconditioning Noisy, Ill-Conditioned Linear Systems James G. Nagy Emory University Atlanta, GA Outline 1. The Basic Problem 2. Regularization / Iterative Methods 3. Preconditioning 4. Example: Image
More information2 Regularized Image Reconstruction for Compressive Imaging and Beyond
EE 367 / CS 448I Computational Imaging and Display Notes: Compressive Imaging and Regularized Image Reconstruction (lecture ) Gordon Wetzstein gordon.wetzstein@stanford.edu This document serves as a supplement
More informationGradient-domain image processing
Gradient-domain image processing http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 10 Course announcements Homework 3 is out. - (Much) smaller
More informationComputational Photography
Computational Photography Si Lu Spring 208 http://web.cecs.pdx.edu/~lusi/cs50/cs50_computati onal_photography.htm 04/0/208 Last Time o Digital Camera History of Camera Controlling Camera o Photography
More informationTaking derivative by convolution
Taking derivative by convolution Partial derivatives with convolution For 2D function f(x,y), the partial derivative is: For discrete data, we can approximate using finite differences: To implement above
More informationOverview. Optimization-Based Data Analysis. Carlos Fernandez-Granda
Overview Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 1/25/2016 Sparsity Denoising Regression Inverse problems Low-rank models Matrix completion
More informationLINEARIZED BREGMAN ITERATIONS FOR FRAME-BASED IMAGE DEBLURRING
LINEARIZED BREGMAN ITERATIONS FOR FRAME-BASED IMAGE DEBLURRING JIAN-FENG CAI, STANLEY OSHER, AND ZUOWEI SHEN Abstract. Real images usually have sparse approximations under some tight frame systems derived
More informationVideo and Motion Analysis Computer Vision Carnegie Mellon University (Kris Kitani)
Video and Motion Analysis 16-385 Computer Vision Carnegie Mellon University (Kris Kitani) Optical flow used for feature tracking on a drone Interpolated optical flow used for super slow-mo optical flow
More informationKernel Correlation for Robust Distance Minimization
Chapter 2 Kernel Correlation for Robust Distance Minimization We introduce kernel correlation between points, between a point and a set of points, and among a set of points. We show that kernel correlation
More informationSparse Sampling: Theory and Applications
November 24, 29 Outline Problem Statement Signals with Finite Rate of Innovation Sampling Kernels: E-splines and -splines Sparse Sampling: the asic Set-up and Extensions The Noisy Scenario Applications
More informationTRACKING and DETECTION in COMPUTER VISION Filtering and edge detection
Technischen Universität München Winter Semester 0/0 TRACKING and DETECTION in COMPUTER VISION Filtering and edge detection Slobodan Ilić Overview Image formation Convolution Non-liner filtering: Median
More informationA Method for Blur and Similarity Transform Invariant Object Recognition
A Method for Blur and Similarity Transform Invariant Object Recognition Ville Ojansivu and Janne Heikkilä Machine Vision Group, Department of Electrical and Information Engineering, University of Oulu,
More informationNonlinear diffusion filtering on extended neighborhood
Applied Numerical Mathematics 5 005) 1 11 www.elsevier.com/locate/apnum Nonlinear diffusion filtering on extended neighborhood Danny Barash Genome Diversity Center, Institute of Evolution, University of
More informationMath 56 Homework 5 Michael Downs
1. (a) Since f(x) = cos(6x) = ei6x 2 + e i6x 2, due to the orthogonality of each e inx, n Z, the only nonzero (complex) fourier coefficients are ˆf 6 and ˆf 6 and they re both 1 2 (which is also seen from
More informationGalaxies in Pennsylvania. Bernstein, Jarvis, Nakajima, & Rusin: Implementation of the BJ02 methods
Galaxies in Pennsylvania Bernstein, Jarvis, Nakajima, & Rusin: Implementation of the BJ02 methods Bases of our methods: Shapes with geometric meaning: No empirical polarizabilities; simple optimizations
More informationA RAIN PIXEL RESTORATION ALGORITHM FOR VIDEOS WITH DYNAMIC SCENES
A RAIN PIXEL RESTORATION ALGORITHM FOR VIDEOS WITH DYNAMIC SCENES V.Sridevi, P.Malarvizhi, P.Mathivannan Abstract Rain removal from a video is a challenging problem due to random spatial distribution and
More informationPHASE UNWRAPPING. Sept. 3, 2007 Lecture D1Lb4 Interferometry: Phase unwrapping Rocca
PHASE UNWRAPPING 1 Phase unwrapping 2 1D Phase unwrapping Problem: given the wrapped phase ψ=w(φ) find the unwrapped one ψ. The wrapping operator: W(φ)=angle(exp(j φ)), gives always solution -π π and is
More informationFrom Fourier Series to Analysis of Non-stationary Signals - II
From Fourier Series to Analysis of Non-stationary Signals - II prof. Miroslav Vlcek October 10, 2017 Contents Signals 1 Signals 2 3 4 Contents Signals 1 Signals 2 3 4 Contents Signals 1 Signals 2 3 4 Contents
More informationLecture 04 Image Filtering
Institute of Informatics Institute of Neuroinformatics Lecture 04 Image Filtering Davide Scaramuzza 1 Lab Exercise 2 - Today afternoon Room ETH HG E 1.1 from 13:15 to 15:00 Work description: your first
More informationFeature extraction: Corners and blobs
Feature extraction: Corners and blobs Review: Linear filtering and edge detection Name two different kinds of image noise Name a non-linear smoothing filter What advantages does median filtering have over
More informationImage Alignment and Mosaicing
Image Alignment and Mosaicing Image Alignment Applications Local alignment: Tracking Stereo Global alignment: Camera jitter elimination Image enhancement Panoramic mosaicing Image Enhancement Original
More informationThe shapes of faint galaxies: A window unto mass in the universe
Lecture 15 The shapes of faint galaxies: A window unto mass in the universe Intensity weighted second moments Optimal filtering Weak gravitational lensing Shear components Shear detection Inverse problem:
More informationNotes on Regularization and Robust Estimation Psych 267/CS 348D/EE 365 Prof. David J. Heeger September 15, 1998
Notes on Regularization and Robust Estimation Psych 67/CS 348D/EE 365 Prof. David J. Heeger September 5, 998 Regularization. Regularization is a class of techniques that have been widely used to solve
More informationERROR COVARIANCE ESTIMATION IN OBJECT TRACKING SCENARIOS USING KALMAN FILTER
ERROR COVARIANCE ESTIMATION IN OBJECT TRACKING SCENARIOS USING KALMAN FILTER Mr K V Sriharsha K.V 1, Dr N V Rao 2 1 Assistant Professor, Department of Information Technology, CVR College of Engg (India)
More informationPreconditioning. Noisy, Ill-Conditioned Linear Systems
Preconditioning Noisy, Ill-Conditioned Linear Systems James G. Nagy Emory University Atlanta, GA Outline 1. The Basic Problem 2. Regularization / Iterative Methods 3. Preconditioning 4. Example: Image
More informationLearning-Based Image Super-Resolution
Limits of Algorithms Learning-Based Image Super-Resolution zhoulin@microsoft.com Microsoft Research Asia Nov. 8, 2008 Limits of Algorithms Outline 1 What is Super-Resolution (SR)? 2 3 Limits of Algorithms
More informationSupplementary Information for cryosparc: Algorithms for rapid unsupervised cryo-em structure determination
Supplementary Information for cryosparc: Algorithms for rapid unsupervised cryo-em structure determination Supplementary Note : Stochastic Gradient Descent (SGD) SGD iteratively optimizes an objective
More informationPart III Super-Resolution with Sparsity
Aisenstadt Chair Course CRM September 2009 Part III Super-Resolution with Sparsity Stéphane Mallat Centre de Mathématiques Appliquées Ecole Polytechnique Super-Resolution with Sparsity Dream: recover high-resolution
More informationEdges and Scale. Image Features. Detecting edges. Origin of Edges. Solution: smooth first. Effects of noise
Edges and Scale Image Features From Sandlot Science Slides revised from S. Seitz, R. Szeliski, S. Lazebnik, etc. Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity
More informationAffine Structure From Motion
EECS43-Advanced Computer Vision Notes Series 9 Affine Structure From Motion Ying Wu Electrical Engineering & Computer Science Northwestern University Evanston, IL 68 yingwu@ece.northwestern.edu Contents
More information10. Multi-objective least squares
L Vandenberghe ECE133A (Winter 2018) 10 Multi-objective least squares multi-objective least squares regularized data fitting control estimation and inversion 10-1 Multi-objective least squares we have
More informationTwo-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix
Two-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix René Vidal Stefano Soatto Shankar Sastry Department of EECS, UC Berkeley Department of Computer Sciences, UCLA 30 Cory Hall,
More informationIntroduction to Compressed Sensing
Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral
More informationImage Processing in Astronomy: Current Practice & Challenges Going Forward
Image Processing in Astronomy: Current Practice & Challenges Going Forward Mario Juric University of Washington With thanks to Andy Connolly, Robert Lupton, Ian Sullivan, David Reiss, and the LSST DM Team
More informationA new class of morphological pyramids for multiresolution image analysis
new class of morphological pyramids for multiresolution image analysis Jos B.T.M. Roerdink Institute for Mathematics and Computing Science University of Groningen P.O. Box 800, 9700 V Groningen, The Netherlands
More informationYou should be able to demonstrate and show your understanding of:
OCR B Physics H557 Module 3: Physics in Action You should be able to demonstrate and show your understanding of: 3.1: Communication 3.1.1: Imaging and Signalling The formation of a real image by a thin
More informationLinear Inverse Problems
Linear Inverse Problems Ajinkya Kadu Utrecht University, The Netherlands February 26, 2018 Outline Introduction Least-squares Reconstruction Methods Examples Summary Introduction 2 What are inverse problems?
More information1. Abstract. 2. Introduction/Problem Statement
Advances in polarimetric deconvolution Capt. Kurtis G. Engelson Air Force Institute of Technology, Student Dr. Stephen C. Cain Air Force Institute of Technology, Professor 1. Abstract One of the realities
More informationSuper-Resolution of Point Sources via Convex Programming
Super-Resolution of Point Sources via Convex Programming Carlos Fernandez-Granda July 205; Revised December 205 Abstract We consider the problem of recovering a signal consisting of a superposition of
More informationRobust Multichannel Blind Deconvolution via Fast Alternating Minimization
Robust Multichannel Blind Deconvolution via Fast Alternating Minimization Filip Šroubek, Member, IEEE, and Peyman Milanfar, Fellow, IEEE Abstract Blind deconvolution, which comprises simultaneous blur
More informationCamera calibration Triangulation
Triangulation Perspective projection in homogenous coordinates ~x img I 0 apple R t 0 T 1 ~x w ~x img R t ~x w Matrix transformations in 2D ~x img K R t ~x w K = 2 3 1 0 t u 40 1 t v 5 0 0 1 Translation
More informationTotal Variation Blind Deconvolution: The Devil is in the Details Technical Report
Total Variation Blind Deconvolution: The Devil is in the Details Technical Report Daniele Perrone University of Bern Bern, Switzerland perrone@iam.unibe.ch Paolo Favaro University of Bern Bern, Switzerland
More information6 The SVD Applied to Signal and Image Deblurring
6 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an
More informationMultiresolution schemes
Multiresolution schemes Fondamenti di elaborazione del segnale multi-dimensionale Multi-dimensional signal processing Stefano Ferrari Università degli Studi di Milano stefano.ferrari@unimi.it Elaborazione
More informationLecture 8: Interest Point Detection. Saad J Bedros
#1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Review of Edge Detectors #2 Today s Lecture Interest Points Detection What do we mean with Interest Point Detection in an Image Goal:
More informationCompression at the source for digital camcorders.
Compression at the source for digital camcorders. Nir Maor, Arie Feuer and Graham C. Goodwin March 7, 2007 Abstract Typical sensors (CCD or CMOS) used in home digital camcorders have the potential of generating
More informationIMPROVING THE DECONVOLUTION METHOD FOR ASTEROID IMAGES: OBSERVING 511 DAVIDA, 52 EUROPA, AND 12 VICTORIA
IMPROVING THE DECONVOLUTION METHOD FOR ASTEROID IMAGES: OBSERVING 511 DAVIDA, 52 EUROPA, AND 12 VICTORIA Z Robert Knight Department of Physics and Astronomy University of Hawai`i at Hilo ABSTRACT Deconvolution
More informationVID3: Sampling and Quantization
Video Transmission VID3: Sampling and Quantization By Prof. Gregory D. Durgin copyright 2009 all rights reserved Claude E. Shannon (1916-2001) Mathematician and Electrical Engineer Worked for Bell Labs
More information8 The SVD Applied to Signal and Image Deblurring
8 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an
More information8 The SVD Applied to Signal and Image Deblurring
8 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an
More informationMultiresolution schemes
Multiresolution schemes Fondamenti di elaborazione del segnale multi-dimensionale Stefano Ferrari Università degli Studi di Milano stefano.ferrari@unimi.it Elaborazione dei Segnali Multi-dimensionali e
More informationSparse Subspace Clustering
Sparse Subspace Clustering Based on Sparse Subspace Clustering: Algorithm, Theory, and Applications by Elhamifar and Vidal (2013) Alex Gutierrez CSCI 8314 March 2, 2017 Outline 1 Motivation and Background
More informationDegeneracies, Dependencies and their Implications in Multi-body and Multi-Sequence Factorizations
Degeneracies, Dependencies and their Implications in Multi-body and Multi-Sequence Factorizations Lihi Zelnik-Manor Michal Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science
More informationSingle-Image-Based Rain and Snow Removal Using Multi-guided Filter
Single-Image-Based Rain and Snow Removal Using Multi-guided Filter Xianhui Zheng 1, Yinghao Liao 1,,WeiGuo 2, Xueyang Fu 2, and Xinghao Ding 2 1 Department of Electronic Engineering, Xiamen University,
More informationOptimization-based sparse recovery: Compressed sensing vs. super-resolution
Optimization-based sparse recovery: Compressed sensing vs. super-resolution Carlos Fernandez-Granda, Google Computational Photography and Intelligent Cameras, IPAM 2/5/2014 This work was supported by a
More informationTutorial 9 The Discrete Fourier Transform (DFT) SIPC , Spring 2017 Technion, CS Department
Tutorial 9 The Discrete Fourier Transform (DFT) SIPC 236327, Spring 2017 Technion, CS Department The DFT Matrix The DFT matrix of size M M is defined as DFT = 1 M W 0 0 W 0 W 0 W where W = e i2π M i =
More informationSensing systems limited by constraints: physical size, time, cost, energy
Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original
More information1 Sparsity and l 1 relaxation
6.883 Learning with Combinatorial Structure Note for Lecture 2 Author: Chiyuan Zhang Sparsity and l relaxation Last time we talked about sparsity and characterized when an l relaxation could recover the
More information