Learning-Based Image Super-Resolution

Size: px
Start display at page:

Download "Learning-Based Image Super-Resolution"

Transcription

1 Limits of Algorithms Learning-Based Image Super-Resolution Microsoft Research Asia Nov. 8, 2008

2 Limits of Algorithms Outline 1 What is Super-Resolution (SR)? 2 3 Limits of Algorithms

3 Limits of Algorithms What is Super-Resolution? SR is a technique that increases image/video details SR vs. Interpolation and Enhancement

4 Limits of Algorithms Classification of SR Algorithms (1) Interpolation-based: register + interpolate + deblur Frequency-based: dealias

5 Limits of Algorithms Classification of SR Algorithms (2) Reconstruction-based: register + weak prior + solve a linear system Learning-based: knowledge + inference

6 Limits of Algorithms Advantages & Disadvantages of + Require fewer low-res images, even single image! + Achieve higher magnification factors (MF) + Faster + More versatile, e.g., style transfer Work with fixed MFs Performance unpredictable

7 Limits of Algorithms Classification of Algorithms (1) Based on applications: For general images For specific images only face/text images

8 Limits of Algorithms Classification of Algorithms (2) Based on implementations: Indirect maximum a posteriori (MAP) ( { } ) N H = arg max P ) Li H i=1 Ḣ P (Ḣ Local: Infer the HR image patch by patch Global: Infer the coefficients of the bases for the HR image Direct MAP ( H = arg max P { } N Ḣ H Li i=1 ) Local only

9 Limits of Algorithms (1) Local indirect maximum a posteriori (MAP): Freeman & Pasztor. Learning Low-Level Vision. ICCV ) P ( L Ĥ = k H = L + Ĥ ) ) Ĥ = arg max P ( L Ĥ P (Ĥ P ( Lk Ĥk Ĥ ) ), P (Ĥ = Ĥ j N (Ĥi ) ) P (Ĥi Ĥj

10 Limits of Algorithms Exemplar Result

11 Limits of Algorithms (2) Global indirect MAP: face hallucination only Gunturk et al. Eigenface-Domain Super-Resolution for Face Recognition. IEEE T. Image Processing, ( ) h = arg max P {l i } N i=1 h P (h) h ( ) ( ) P {l i } N N i=1 h exp ξi t Q 1 ξ i, ξ i = l i F t l P(i) F h h η P (h) exp i=1 ( ) (h µ h ) t Λ 1 (h µ h )

12 Limits of Algorithms Exemplar Result

13 Limits of Algorithms (3) Local direct MAP: Sun, Tao, and Shum. Image Hallucination with Primal Sketch Priors. CVPR H = L + H p H p = arg max P ( H p L ) H p P ( H p L ) k P(C k L), P(C k L) n k 1 i n k Ψ(Bi l, Bl i+1 ) Φ(Bi l, Bh i ) i

14 Limits of Algorithms Exemplar Result

15 Limits of Algorithms (4) Using manifold learning techniques: Chang, Yeung, and Xiong. Super-Resolution Through Neighbor Embedding. CVPR For each LR patch 2 Enforcing local compatibility and smoothness constraints between adjacent HR patches.

16 Limits of Algorithms Exemplar Result

17 Limits of Algorithms Do limits exist for?

18 Limits of Algorithms Related Work Limits of Reconstruction-Based SR Algorithms [2]

19 Limits of Algorithms What are limits of SR? Good SR result: close to the ground truth Average performance

20 Limits of Algorithms Abstract Model of An SR Algorithm: a mapping s from LR image (low-dim space) to HR image (high-dim space) Average performance: expected risk R(s) = r(h, s(d(h)))p(h)dh r: risk function, d: downsampling operator, p(h): distribution

21 Limits of Algorithms Problem Formulation R(s) = g s (N, m) = g s (N, m) = h ( ) 1 1/2 mn g s (N, m) h s(dh) 2 p h (h)dh N: image size, m: magnification factor, D: downsampling matrix Does not help if compute g s (N, m) for a particular s. Find lower bound b(n, m) for g s (N, m) that is valid for all s. Lower bound is indefinite if no assumption on p h (h) is made.

22 Limits of Algorithms Statistics of General Natural Images The distribution of HR images (HRI) is not concentrated around several HRIs, and the distribution of LR images (LRI) is not concentrated around several LRIs either. Smoother LRIs have a higher probability than nonsmooth ones. Statistics of specific class of images is unclear.

23 Limits of Algorithms Theorem 1: Lower Bound of the Expected Risk g s (N, m) = h h s(dh) 2 p h (h)dh Theorem 1: g s (N, m) is lower bounded by b(n, m), where b(n, m) = 1 4 tr [ (I UD)Σ(I UD) t] (I UD) h 4 U: upsampling matrix, DU = I Σ: covariance matrix, h: mean

24 Limits of Algorithms Sketch of Proof (1) g s (N, m) = Choose Q and V such that M = (U h h s(dh) 2 p h (h)dh ( D Q V). Perform transform h = M ) (U V) = I. Denote ( ) x, then y

25 Limits of Algorithms Sketch of Proof (2) g(n, m) = = x,y x (U V) ( x y p x (x)v (x)dx, ) s (x) 2 p x,y (( x y (( )) ( ( )) x x p x,y = M p y h M y V (x) = Vy φ (x) 2 p y (y x) dy y φ(x) = s(x) Ux )) dxdy

26 Limits of Algorithms Sketch of Proof (3) V (x) = V (x) = y Vy φ (x) 2 p y (y x) dy φ opt (x; p y ) = V y φ opt (x; p y ) y y p y (y x) dy Vy 2 p y (y x) dy φopt (x; p y ) 2 y Vy 2 p x,y (( x y p x (x) )) dy

27 Limits of Algorithms Sketch of Proof (4) g(n, m) = x 1 4 x = 4 1 = 1 4 p x (x) x,y h y p x (x) y Vy 2 p y (y x) dy φ opt (x; p y ) 2 dx Vy 2 p x,y (( x y VQh 2 p h (h)dh Vy 2 p y (y x) dydx )) dxdy = 1 4 tr ( (I UD)Σ(I UD) t) (I UD) h 2

28 Limits of Algorithms Estimating the Lower Bound via Sampling Theorem 2: If we sample M(p, ε) = (C 1 + 2C 2 ) 2 16pε 2 HRIs independently, then with probability of at least 1 p, ˆ b(n, m) b(n, m) < ε. ˆ b(n, m) is the value of b(n, m) estimated from real samples, ( C 1 = E (I UD)(h h) ) 4 tr 2 [(I UD)Σ(I UD) t ], and C 2 = bt Σ b, b = (I UD) t (I UD) h.

29 Limits of Algorithms Sketch of Proof (1) 1 ˆ b(n, m) b(n, m) 4 ξ Eξ + 1 η Eη 2 ˆ Σ M = 1 M ξ = tr(bˆ Σ M ), M (ĥk h)(ĥk h) t, k=1 B = (I UD) t (I UD), t η = b ˆ hm ˆ h M = 1 M b = B h M k=1 ĥ k

30 Limits of Algorithms Sketch of Proof (2) 1 ˆ b(n, m) b(n, m) 4 ξ Eξ + 1 η Eη 2 P( ξ Eξ δ) varξ δ 2 = C2 1 Mδ 2 P( η Eη δ) varη δ 2 = C2 2 Mδ 2 So with probability at least 1 p, ˆ b(n, m) b(n, m) C 1 4 Mp + C 2 2 Mp

31 Limits of Algorithms Estimating the Limits via the Lower Bound If at a particular MF m, b(n, m) is larger than a threshold T, then at this MF no SR algorithm can effectively recover the original HRI: limit b 1 (T )

32 Limits of Algorithms Experiments

33 Limits of Algorithms Estimating the Limits T = 11.1 is a large enough threshold.

34 Limits of Algorithms Considering the Noise To take noise into account, g(n, m) should be changed to (( )) g (N, m) = h s (Dh + n) 2 h p h,n dhdn, n Accordingly, h,n b (N, m) = 4 1tr [ (I UD)Σ(I UD) t] tr ( UΣ n U t) (I UD) h U n 2.

35 Limits of Algorithms Future Work and Open Problems Tighter upper bound of the limits Limits of SR algorithms for specific image classes How to represent and incorporate the prior more effectively? How to make the algorithms scalable with the MF? What is the relationship between the SR performance and the training samples? How to choose optimal training samples?

36 Limits of Algorithms References et al. Limits of Learning-Based Superresolution Algorithms. Int l J. Computer Vision, et al. Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation, IEEE T. PAMI, 2004.

37 Limits of Algorithms Questions?

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials by Phillip Krahenbuhl and Vladlen Koltun Presented by Adam Stambler Multi-class image segmentation Assign a class label to each

More information

Motion Estimation (I) Ce Liu Microsoft Research New England

Motion Estimation (I) Ce Liu Microsoft Research New England Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

1 Inria Rennes - Bretagne Atlantique 2 Alcatel-Lucent - Bell Labs France

1 Inria Rennes - Bretagne Atlantique 2 Alcatel-Lucent - Bell Labs France LOW-COMPLEXITY SINGLE-IMAGE SUPER-RESOLUTION BASED ON NONNEGATIVE NEIGHBOR EMBEDDING Marco Bevilacqua 1,2, Aline Roumy 1, Christine Guillemot 1, Marie-Line Alberi Morel 2 1 Inria Rennes - Bretagne Atlantique

More information

DETECTING HUMAN ACTIVITIES IN THE ARCTIC OCEAN BY CONSTRUCTING AND ANALYZING SUPER-RESOLUTION IMAGES FROM MODIS DATA INTRODUCTION

DETECTING HUMAN ACTIVITIES IN THE ARCTIC OCEAN BY CONSTRUCTING AND ANALYZING SUPER-RESOLUTION IMAGES FROM MODIS DATA INTRODUCTION DETECTING HUMAN ACTIVITIES IN THE ARCTIC OCEAN BY CONSTRUCTING AND ANALYZING SUPER-RESOLUTION IMAGES FROM MODIS DATA Shizhi Chen and YingLi Tian Department of Electrical Engineering The City College of

More information

Super-Resolution. Shai Avidan Tel-Aviv University

Super-Resolution. Shai Avidan Tel-Aviv University Super-Resolution Shai Avidan Tel-Aviv University Slide Credits (partial list) Ric Szelisi Steve Seitz Alyosha Efros Yacov Hel-Or Yossi Rubner Mii Elad Marc Levoy Bill Freeman Fredo Durand Sylvain Paris

More information

Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi

Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Overview Introduction Linear Methods for Dimensionality Reduction Nonlinear Methods and Manifold

More information

CS 4495 Computer Vision Principle Component Analysis

CS 4495 Computer Vision Principle Component Analysis CS 4495 Computer Vision Principle Component Analysis (and it s use in Computer Vision) Aaron Bobick School of Interactive Computing Administrivia PS6 is out. Due *** Sunday, Nov 24th at 11:55pm *** PS7

More information

Covariance-Based PCA for Multi-Size Data

Covariance-Based PCA for Multi-Size Data Covariance-Based PCA for Multi-Size Data Menghua Zhai, Feiyu Shi, Drew Duncan, and Nathan Jacobs Department of Computer Science, University of Kentucky, USA {mzh234, fsh224, drew, jacobs}@cs.uky.edu Abstract

More information

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems)

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Donghwan Kim and Jeffrey A. Fessler EECS Department, University of Michigan

More information

Motion Estimation (I)

Motion Estimation (I) Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

Super-Resolution. Dr. Yossi Rubner. Many slides from Miki Elad - Technion

Super-Resolution. Dr. Yossi Rubner. Many slides from Miki Elad - Technion Super-Resolution Dr. Yossi Rubner yossi@rubner.co.il Many slides from Mii Elad - Technion 5/5/2007 53 images, ratio :4 Example - Video 40 images ratio :4 Example Surveillance Example Enhance Mosaics Super-Resolution

More information

Resolving the White Noise Paradox in the Regularisation of Inverse Problems

Resolving the White Noise Paradox in the Regularisation of Inverse Problems 1 / 32 Resolving the White Noise Paradox in the Regularisation of Inverse Problems Hanne Kekkonen joint work with Matti Lassas and Samuli Siltanen Department of Mathematics and Statistics University of

More information

Face detection and recognition. Detection Recognition Sally

Face detection and recognition. Detection Recognition Sally Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification

More information

Reconnaissance d objetsd et vision artificielle

Reconnaissance d objetsd et vision artificielle Reconnaissance d objetsd et vision artificielle http://www.di.ens.fr/willow/teaching/recvis09 Lecture 6 Face recognition Face detection Neural nets Attention! Troisième exercice de programmation du le

More information

Robust exponential convergence of hp-fem for singularly perturbed systems of reaction-diffusion equations

Robust exponential convergence of hp-fem for singularly perturbed systems of reaction-diffusion equations Robust exponential convergence of hp-fem for singularly perturbed systems of reaction-diffusion equations Christos Xenophontos Department of Mathematics and Statistics University of Cyprus joint work with

More information

Supervised Deep Kriging for Single-Image Super-Resolution

Supervised Deep Kriging for Single-Image Super-Resolution Supervised Deep Kriging for Single-Image Super-Resolution Gianni FRANCHI 1, Angela YAO 2, and Andreas KOLB 1 1 Institute for Vision and Graphics, University of Siegen, Germany {gianni.franchi,andreas.kolb}@uni-siegen.de

More information

Asaf Bar Zvi Adi Hayat. Semantic Segmentation

Asaf Bar Zvi Adi Hayat. Semantic Segmentation Asaf Bar Zvi Adi Hayat Semantic Segmentation Today s Topics Fully Convolutional Networks (FCN) (CVPR 2015) Conditional Random Fields as Recurrent Neural Networks (ICCV 2015) Gaussian Conditional random

More information

Edge Detection. Computer Vision P. Schrater Spring 2003

Edge Detection. Computer Vision P. Schrater Spring 2003 Edge Detection Computer Vision P. Schrater Spring 2003 Simplest Model: (Canny) Edge(x) = a U(x) + n(x) U(x)? x=0 Convolve image with U and find points with high magnitude. Choose value by comparing with

More information

MIRA, SVM, k-nn. Lirong Xia

MIRA, SVM, k-nn. Lirong Xia MIRA, SVM, k-nn Lirong Xia Linear Classifiers (perceptrons) Inputs are feature values Each feature has a weight Sum is the activation activation w If the activation is: Positive: output +1 Negative, output

More information

Toward a Realization of Marr s Theory of Primal Sketches via Autocorrelation Wavelets: Image Representation using Multiscale Edge Information

Toward a Realization of Marr s Theory of Primal Sketches via Autocorrelation Wavelets: Image Representation using Multiscale Edge Information Toward a Realization of Marr s Theory of Primal Sketches via Autocorrelation Wavelets: Image Representation using Multiscale Edge Information Naoki Saito 1 Department of Mathematics University of California,

More information

MAP Examples. Sargur Srihari

MAP Examples. Sargur Srihari MAP Examples Sargur srihari@cedar.buffalo.edu 1 Potts Model CRF for OCR Topics Image segmentation based on energy minimization 2 Examples of MAP Many interesting examples of MAP inference are instances

More information

Deep Learning: Approximation of Functions by Composition

Deep Learning: Approximation of Functions by Composition Deep Learning: Approximation of Functions by Composition Zuowei Shen Department of Mathematics National University of Singapore Outline 1 A brief introduction of approximation theory 2 Deep learning: approximation

More information

DEPARTMENT OF COMPUTER SCIENCE Autumn Semester MACHINE LEARNING AND ADAPTIVE INTELLIGENCE

DEPARTMENT OF COMPUTER SCIENCE Autumn Semester MACHINE LEARNING AND ADAPTIVE INTELLIGENCE Data Provided: None DEPARTMENT OF COMPUTER SCIENCE Autumn Semester 203 204 MACHINE LEARNING AND ADAPTIVE INTELLIGENCE 2 hours Answer THREE of the four questions. All questions carry equal weight. Figures

More information

Eigenface-based facial recognition

Eigenface-based facial recognition Eigenface-based facial recognition Dimitri PISSARENKO December 1, 2002 1 General This document is based upon Turk and Pentland (1991b), Turk and Pentland (1991a) and Smith (2002). 2 How does it work? The

More information

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 THE ROAL STATISTICAL SOCIET 6 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE The Society is providing these solutions to assist candidates preparing for the examinations in 7. The solutions are intended

More information

Cross-Validation with Confidence

Cross-Validation with Confidence Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University UMN Statistics Seminar, Mar 30, 2017 Overview Parameter est. Model selection Point est. MLE, M-est.,... Cross-validation

More information

Outline. Spring It Introduction Representation. Markov Random Field. Conclusion. Conditional Independence Inference: Variable elimination

Outline. Spring It Introduction Representation. Markov Random Field. Conclusion. Conditional Independence Inference: Variable elimination Probabilistic Graphical Models COMP 790-90 Seminar Spring 2011 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline It Introduction ti Representation Bayesian network Conditional Independence Inference:

More information

PCA and LDA. Man-Wai MAK

PCA and LDA. Man-Wai MAK PCA and LDA Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: S.J.D. Prince,Computer

More information

Advances in Manifold Learning Presented by: Naku Nak l Verm r a June 10, 2008

Advances in Manifold Learning Presented by: Naku Nak l Verm r a June 10, 2008 Advances in Manifold Learning Presented by: Nakul Verma June 10, 008 Outline Motivation Manifolds Manifold Learning Random projection of manifolds for dimension reduction Introduction to random projections

More information

(d). Why does this imply that there is no bounded extension operator E : W 1,1 (U) W 1,1 (R n )? Proof. 2 k 1. a k 1 a k

(d). Why does this imply that there is no bounded extension operator E : W 1,1 (U) W 1,1 (R n )? Proof. 2 k 1. a k 1 a k Exercise For k 0,,... let k be the rectangle in the plane (2 k, 0 + ((0, (0, and for k, 2,... let [3, 2 k ] (0, ε k. Thus is a passage connecting the room k to the room k. Let ( k0 k (. We assume ε k

More information

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Philipp Krähenbühl and Vladlen Koltun Stanford University Presenter: Yuan-Ting Hu 1 Conditional Random Field (CRF) E x I = φ u

More information

The integral test and estimates of sums

The integral test and estimates of sums The integral test Suppose f is a continuous, positive, decreasing function on [, ) and let a n = f (n). Then the series n= a n is convergent if and only if the improper integral f (x)dx is convergent.

More information

An Adaptive Test of Independence with Analytic Kernel Embeddings

An Adaptive Test of Independence with Analytic Kernel Embeddings An Adaptive Test of Independence with Analytic Kernel Embeddings Wittawat Jitkrittum 1 Zoltán Szabó 2 Arthur Gretton 1 1 Gatsby Unit, University College London 2 CMAP, École Polytechnique ICML 2017, Sydney

More information

D. Correct! This is the correct answer. It is found by dy/dx = (dy/dt)/(dx/dt).

D. Correct! This is the correct answer. It is found by dy/dx = (dy/dt)/(dx/dt). Calculus II - Problem Solving Drill 4: Calculus for Parametric Equations Question No. of 0 Instructions: () Read the problem and answer choices carefully () Work the problems on paper as. Find dy/dx where

More information

Classification objectives COMS 4771

Classification objectives COMS 4771 Classification objectives COMS 4771 1. Recap: binary classification Scoring functions Consider binary classification problems with Y = { 1, +1}. 1 / 22 Scoring functions Consider binary classification

More information

Unlabeled Data: Now It Helps, Now It Doesn t

Unlabeled Data: Now It Helps, Now It Doesn t institution-logo-filena A. Singh, R. D. Nowak, and X. Zhu. In NIPS, 2008. 1 Courant Institute, NYU April 21, 2015 Outline institution-logo-filena 1 Conflicting Views in Semi-supervised Learning The Cluster

More information

Optimal global rates of convergence for interpolation problems with random design

Optimal global rates of convergence for interpolation problems with random design Optimal global rates of convergence for interpolation problems with random design Michael Kohler 1 and Adam Krzyżak 2, 1 Fachbereich Mathematik, Technische Universität Darmstadt, Schlossgartenstr. 7, 64289

More information

Dictionary Learning for photo-z estimation

Dictionary Learning for photo-z estimation Dictionary Learning for photo-z estimation Joana Frontera-Pons, Florent Sureau, Jérôme Bobin 5th September 2017 - Workshop Dictionary Learning on Manifolds MOTIVATION! Goal : Measure the radial positions

More information

Dimension Reduction Techniques. Presented by Jie (Jerry) Yu

Dimension Reduction Techniques. Presented by Jie (Jerry) Yu Dimension Reduction Techniques Presented by Jie (Jerry) Yu Outline Problem Modeling Review of PCA and MDS Isomap Local Linear Embedding (LLE) Charting Background Advances in data collection and storage

More information

FACE RECOGNITION USING CCA ON NONLINEAR FEATURES

FACE RECOGNITION USING CCA ON NONLINEAR FEATURES FACE RECOGNITION USING CCA ON NONLINEAR FEATURES N. Narmada 1, S. Aruna Mastani 2 1 M.Tech (DECS) & 2 Assistant Professor Department of ECE, JNTU College of Engineering, Anantapur, Andhra Pradesh, India

More information

Regularization Theory

Regularization Theory Regularization Theory Solving the inverse problem of Super resolution with CNN Aditya Ganeshan Under the guidance of Dr. Ankik Kumar Giri December 13, 2016 Table of Content 1 Introduction Material coverage

More information

A GENERAL FORMULATION FOR SUPPORT VECTOR MACHINES. Wei Chu, S. Sathiya Keerthi, Chong Jin Ong

A GENERAL FORMULATION FOR SUPPORT VECTOR MACHINES. Wei Chu, S. Sathiya Keerthi, Chong Jin Ong A GENERAL FORMULATION FOR SUPPORT VECTOR MACHINES Wei Chu, S. Sathiya Keerthi, Chong Jin Ong Control Division, Department of Mechanical Engineering, National University of Singapore 0 Kent Ridge Crescent,

More information

Statistical Analysis of Tensor Fields

Statistical Analysis of Tensor Fields Statistical Analysis of Tensor Fields Yuchen Xie Baba C. Vemuri Jeffrey Ho Department of Computer and Information Sciences and Engineering University of Florida Abstract. In this paper, we propose a Riemannian

More information

Morphing. Xiao-Ming Fu

Morphing. Xiao-Ming Fu Morphing Xiao-Ming Fu Outlines Definition Angle, length, area, volume, and curvature Example-Driven Deformations Based on Discrete Shells Affine transformation As-Rigid-As-Possible Shape Interpolation

More information

Methods for sparse analysis of high-dimensional data, II

Methods for sparse analysis of high-dimensional data, II Methods for sparse analysis of high-dimensional data, II Rachel Ward May 23, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 47 High dimensional

More information

i=1 α i. Given an m-times continuously

i=1 α i. Given an m-times continuously 1 Fundamentals 1.1 Classification and characteristics Let Ω R d, d N, d 2, be an open set and α = (α 1,, α d ) T N d 0, N 0 := N {0}, a multiindex with α := d i=1 α i. Given an m-times continuously differentiable

More information

The Power of Asymmetry in Binary Hashing

The Power of Asymmetry in Binary Hashing The Power of Asymmetry in Binary Hashing Behnam Neyshabur Yury Makarychev Toyota Technological Institute at Chicago Russ Salakhutdinov University of Toronto Nati Srebro Technion/TTIC Search by Image Image

More information

Cross-Validation with Confidence

Cross-Validation with Confidence Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University WHOA-PSI Workshop, St Louis, 2017 Quotes from Day 1 and Day 2 Good model or pure model? Occam s razor We really

More information

Lecture: Face Recognition

Lecture: Face Recognition Lecture: Face Recognition Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 12-1 What we will learn today Introduction to face recognition The Eigenfaces Algorithm Linear

More information

Classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Topics Discriminant functions Logistic regression Perceptron Generative models Generative vs. discriminative

More information

MEI STRUCTURED MATHEMATICS 4753/1

MEI STRUCTURED MATHEMATICS 4753/1 OXFORD CAMBRIDGE AND RSA EXAMINATIONS Advanced Subsidiary General Certificate of Education Advanced General Certificate of Education MEI STRUCTURED MATHEMATICS 473/ Methods for Advanced Mathematics (C3)

More information

Nonlinear Dimensionality Reduction

Nonlinear Dimensionality Reduction Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Kernel PCA 2 Isomap 3 Locally Linear Embedding 4 Laplacian Eigenmap

More information

Distributed Inexact Newton-type Pursuit for Non-convex Sparse Learning

Distributed Inexact Newton-type Pursuit for Non-convex Sparse Learning Distributed Inexact Newton-type Pursuit for Non-convex Sparse Learning Bo Liu Department of Computer Science, Rutgers Univeristy Xiao-Tong Yuan BDAT Lab, Nanjing University of Information Science and Technology

More information

1 The Continuous Wavelet Transform The continuous wavelet transform (CWT) Discretisation of the CWT... 2

1 The Continuous Wavelet Transform The continuous wavelet transform (CWT) Discretisation of the CWT... 2 Contents 1 The Continuous Wavelet Transform 1 1.1 The continuous wavelet transform (CWT)............. 1 1. Discretisation of the CWT...................... Stationary wavelet transform or redundant wavelet

More information

Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors

Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors Jan Kotera 1,2, Filip Šroubek1, and Peyman Milanfar 3 1 Institue of Information Theory and Automation, Academy

More information

Level-Set Person Segmentation and Tracking with Multi-Region Appearance Models and Top-Down Shape Information Appendix

Level-Set Person Segmentation and Tracking with Multi-Region Appearance Models and Top-Down Shape Information Appendix Level-Set Person Segmentation and Tracking with Multi-Region Appearance Models and Top-Down Shape Information Appendix Esther Horbert, Konstantinos Rematas, Bastian Leibe UMIC Research Centre, RWTH Aachen

More information

Lecture 19: Bayesian Linear Estimators

Lecture 19: Bayesian Linear Estimators ECE 830 Fall 2010 Statistical Signal Processing instructor: R Nowa, scribe: I Rosado-Mendez Lecture 19: Bayesian Linear Estimators 1 Linear Minimum Mean-Square Estimator Suppose our data is set X R n,

More information

Single-Image-Based Rain and Snow Removal Using Multi-guided Filter

Single-Image-Based Rain and Snow Removal Using Multi-guided Filter Single-Image-Based Rain and Snow Removal Using Multi-guided Filter Xianhui Zheng 1, Yinghao Liao 1,,WeiGuo 2, Xueyang Fu 2, and Xinghao Ding 2 1 Department of Electronic Engineering, Xiamen University,

More information

A Modified Incremental Principal Component Analysis for On-line Learning of Feature Space and Classifier

A Modified Incremental Principal Component Analysis for On-line Learning of Feature Space and Classifier A Modified Incremental Principal Component Analysis for On-line Learning of Feature Space and Classifier Seiichi Ozawa, Shaoning Pang, and Nikola Kasabov Graduate School of Science and Technology, Kobe

More information

Classifier Complexity and Support Vector Classifiers

Classifier Complexity and Support Vector Classifiers Classifier Complexity and Support Vector Classifiers Feature 2 6 4 2 0 2 4 6 8 RBF kernel 10 10 8 6 4 2 0 2 4 6 Feature 1 David M.J. Tax Pattern Recognition Laboratory Delft University of Technology D.M.J.Tax@tudelft.nl

More information

Scalable algorithms for optimal experimental design for infinite-dimensional nonlinear Bayesian inverse problems

Scalable algorithms for optimal experimental design for infinite-dimensional nonlinear Bayesian inverse problems Scalable algorithms for optimal experimental design for infinite-dimensional nonlinear Bayesian inverse problems Alen Alexanderian (Math/NC State), Omar Ghattas (ICES/UT-Austin), Noémi Petra (Applied Math/UC

More information

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 BASEL. Logistic Regression. Pattern Recognition 2016 Sandro Schönborn University of Basel

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 BASEL. Logistic Regression. Pattern Recognition 2016 Sandro Schönborn University of Basel Logistic Regression Pattern Recognition 2016 Sandro Schönborn University of Basel Two Worlds: Probabilistic & Algorithmic We have seen two conceptual approaches to classification: data class density estimation

More information

Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors

Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Sean Borman and Robert L. Stevenson Department of Electrical Engineering, University of Notre Dame Notre Dame,

More information

SYDE 575: Introduction to Image Processing. Image Compression Part 2: Variable-rate compression

SYDE 575: Introduction to Image Processing. Image Compression Part 2: Variable-rate compression SYDE 575: Introduction to Image Processing Image Compression Part 2: Variable-rate compression Variable-rate Compression: Transform-based compression As mentioned earlier, we wish to transform image data

More information

Autoencoders and Score Matching. Based Models. Kevin Swersky Marc Aurelio Ranzato David Buchman Benjamin M. Marlin Nando de Freitas

Autoencoders and Score Matching. Based Models. Kevin Swersky Marc Aurelio Ranzato David Buchman Benjamin M. Marlin Nando de Freitas On for Energy Based Models Kevin Swersky Marc Aurelio Ranzato David Buchman Benjamin M. Marlin Nando de Freitas Toronto Machine Learning Group Meeting, 2011 Motivation Models Learning Goal: Unsupervised

More information

Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr

Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters

More information

arxiv: v1 [math.na] 1 May 2013

arxiv: v1 [math.na] 1 May 2013 arxiv:3050089v [mathna] May 03 Approximation Properties of a Gradient Recovery Operator Using a Biorthogonal System Bishnu P Lamichhane and Adam McNeilly May, 03 Abstract A gradient recovery operator based

More information

Review: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF)

Review: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF) Case Study 4: Collaborative Filtering Review: Probabilistic Matrix Factorization Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox February 2 th, 214 Emily Fox 214 1 Probabilistic

More information

Image Processing 2. Hakan Bilen University of Edinburgh. Computer Graphics Fall 2017

Image Processing 2. Hakan Bilen University of Edinburgh. Computer Graphics Fall 2017 Image Processing 2 Hakan Bilen University of Edinburgh Computer Graphics Fall 2017 This week What is an image? What is image processing? Point processing Linear (Spatial) filters Frequency domain Deep

More information

Gaussian with mean ( µ ) and standard deviation ( σ)

Gaussian with mean ( µ ) and standard deviation ( σ) Slide from Pieter Abbeel Gaussian with mean ( µ ) and standard deviation ( σ) 10/6/16 CSE-571: Robotics X ~ N( µ, σ ) Y ~ N( aµ + b, a σ ) Y = ax + b + + + + 1 1 1 1 1 1 1 1 1 1, ~ ) ( ) ( ), ( ~ ), (

More information

An Introduction to Filterbank Frames

An Introduction to Filterbank Frames An Introduction to Filterbank Frames Brody Dylan Johnson St. Louis University October 19, 2010 Brody Dylan Johnson (St. Louis University) An Introduction to Filterbank Frames October 19, 2010 1 / 34 Overview

More information

No. of dimensions 1. No. of centers

No. of dimensions 1. No. of centers Contents 8.6 Course of dimensionality............................ 15 8.7 Computational aspects of linear estimators.................. 15 8.7.1 Diagonalization of circulant andblock-circulant matrices......

More information

Self-Tuning Semantic Image Segmentation

Self-Tuning Semantic Image Segmentation Self-Tuning Semantic Image Segmentation Sergey Milyaev 1,2, Olga Barinova 2 1 Voronezh State University sergey.milyaev@gmail.com 2 Lomonosov Moscow State University obarinova@graphics.cs.msu.su Abstract.

More information

Methods for sparse analysis of high-dimensional data, II

Methods for sparse analysis of high-dimensional data, II Methods for sparse analysis of high-dimensional data, II Rachel Ward May 26, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 55 High dimensional

More information

Weighted Collaborative Representation and Classification of Images

Weighted Collaborative Representation and Classification of Images Weighted Collaborative Representation and Classification of Images Radu Timofte 1 and Luc Van Gool 1,2 1 ESAT-PSI/IBBT, Catholic University of Leuven, Belgium 2 D-ITET, ETH Zurich, Switzerland {Radu.Timofte,

More information

Sobolev spaces, Trace theorems and Green s functions.

Sobolev spaces, Trace theorems and Green s functions. Sobolev spaces, Trace theorems and Green s functions. Boundary Element Methods for Waves Scattering Numerical Analysis Seminar. Orane Jecker October 21, 2010 Plan Introduction 1 Useful definitions 2 Distributions

More information

Spectral Gap and Concentration for Some Spherically Symmetric Probability Measures

Spectral Gap and Concentration for Some Spherically Symmetric Probability Measures Spectral Gap and Concentration for Some Spherically Symmetric Probability Measures S.G. Bobkov School of Mathematics, University of Minnesota, 127 Vincent Hall, 26 Church St. S.E., Minneapolis, MN 55455,

More information

A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier

A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier Seiichi Ozawa 1, Shaoning Pang 2, and Nikola Kasabov 2 1 Graduate School of Science and Technology,

More information

L26: Advanced dimensionality reduction

L26: Advanced dimensionality reduction L26: Advanced dimensionality reduction The snapshot CA approach Oriented rincipal Components Analysis Non-linear dimensionality reduction (manifold learning) ISOMA Locally Linear Embedding CSCE 666 attern

More information

EECS 750. Hypothesis Testing with Communication Constraints

EECS 750. Hypothesis Testing with Communication Constraints EECS 750 Hypothesis Testing with Communication Constraints Name: Dinesh Krithivasan Abstract In this report, we study a modification of the classical statistical problem of bivariate hypothesis testing.

More information

Introduction to finite element exterior calculus

Introduction to finite element exterior calculus Introduction to finite element exterior calculus Ragnar Winther CMA, University of Oslo Norway Why finite element exterior calculus? Recall the de Rham complex on the form: R H 1 (Ω) grad H(curl, Ω) curl

More information

Image Analysis & Retrieval. Lec 14. Eigenface and Fisherface

Image Analysis & Retrieval. Lec 14. Eigenface and Fisherface Image Analysis & Retrieval Lec 14 Eigenface and Fisherface Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu Z. Li, Image Analysis & Retrv, Spring

More information

Support Vector Machine (SVM) and Kernel Methods

Support Vector Machine (SVM) and Kernel Methods Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2014 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin

More information

Cutting Plane Training of Structural SVM

Cutting Plane Training of Structural SVM Cutting Plane Training of Structural SVM Seth Neel University of Pennsylvania sethneel@wharton.upenn.edu September 28, 2017 Seth Neel (Penn) Short title September 28, 2017 1 / 33 Overview Structural SVMs

More information

1 Exercises for lecture 1

1 Exercises for lecture 1 1 Exercises for lecture 1 Exercise 1 a) Show that if F is symmetric with respect to µ, and E( X )

More information

6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018

6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 6.869 Advances in Computer Vision Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 1 Sampling Sampling Pixels Continuous world 3 Sampling 4 Sampling 5 Continuous image f (x, y) Sampling

More information

Lecture 13 Visual recognition

Lecture 13 Visual recognition Lecture 13 Visual recognition Announcements Silvio Savarese Lecture 13-20-Feb-14 Lecture 13 Visual recognition Object classification bag of words models Discriminative methods Generative methods Object

More information

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Data Mining Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 42 Outline 1 Introduction 2 Feature selection

More information

Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005

Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Edge Detection PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Gradients and edges Points of sharp change in an image are interesting: change in reflectance change in object change

More information

Manifold Regularization

Manifold Regularization 9.520: Statistical Learning Theory and Applications arch 3rd, 200 anifold Regularization Lecturer: Lorenzo Rosasco Scribe: Hooyoung Chung Introduction In this lecture we introduce a class of learning algorithms,

More information

Non-parametric Methods

Non-parametric Methods Non-parametric Methods Machine Learning Alireza Ghane Non-Parametric Methods Alireza Ghane / Torsten Möller 1 Outline Machine Learning: What, Why, and How? Curve Fitting: (e.g.) Regression and Model Selection

More information

41903: Introduction to Nonparametrics

41903: Introduction to Nonparametrics 41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific

More information

Digital Image Processing. Filtering in the Frequency Domain

Digital Image Processing. Filtering in the Frequency Domain 2D Linear Systems 2D Fourier Transform and its Properties The Basics of Filtering in Frequency Domain Image Smoothing Image Sharpening Selective Filtering Implementation Tips 1 General Definition: System

More information

Linear Models for Regression

Linear Models for Regression Linear Models for Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr

More information

Deriving Principal Component Analysis (PCA)

Deriving Principal Component Analysis (PCA) -0 Mathematical Foundations for Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deriving Principal Component Analysis (PCA) Matt Gormley Lecture 11 Oct.

More information

Estimating Single-Agent Dynamic Models

Estimating Single-Agent Dynamic Models Estimating Single-Agent Dynamic Models Paul T. Scott Empirical IO Fall, 2013 1 / 49 Why are dynamics important? The motivation for using dynamics is usually external validity: we want to simulate counterfactuals

More information

Harris Corner Detector

Harris Corner Detector Multimedia Computing: Algorithms, Systems, and Applications: Feature Extraction By Dr. Yu Cao Department of Computer Science The University of Massachusetts Lowell Lowell, MA 01854, USA Part of the slides

More information

Lecture 14: Deep Generative Learning

Lecture 14: Deep Generative Learning Generative Modeling CSED703R: Deep Learning for Visual Recognition (2017F) Lecture 14: Deep Generative Learning Density estimation Reconstructing probability density function using samples Bohyung Han

More information

Principal Component Analysis

Principal Component Analysis B: Chapter 1 HTF: Chapter 1.5 Principal Component Analysis Barnabás Póczos University of Alberta Nov, 009 Contents Motivation PCA algorithms Applications Face recognition Facial expression recognition

More information

An Adaptive Test of Independence with Analytic Kernel Embeddings

An Adaptive Test of Independence with Analytic Kernel Embeddings An Adaptive Test of Independence with Analytic Kernel Embeddings Wittawat Jitkrittum Gatsby Unit, University College London wittawat@gatsby.ucl.ac.uk Probabilistic Graphical Model Workshop 2017 Institute

More information