Introduction to Mathematical Programming

Transcription

1 Introduction to Mathematical Programming Ming Zhong Lecture 25 November 5, 2018 Ming Zhong (JHU) AMS Fall / 19

2 Table of Contents 1 Ming Zhong (JHU) AMS Fall / 19

3 Some Preliminaries: Fourier Transform Fourier Transform The Fourier Transform (FT) decomposes of a function of time into the frequencies. Definition Given an integrable function f : R C, the Fourier transform gives a function ˆf : R C in the following, ˆf (ξ) = And the inverse transform, f (x) = f (x)e 2πixξ dx, for any ξ R. ˆf (ξ)e 2πixξ dξ, for any x R. Ming Zhong (JHU) AMS Fall / 19

4 Fourier Transform, cont. Furthermore, In 1822, Joseph Fourier showed that some functions could be written as an infinite sum of harmonics. Let f, g, h be integrable functions, f (x) dx <, then If h(x) = af (x) + bg(x), then ĥ(ξ) = aˆf (ξ) + bĝ(ξ). If h(x) = f (x x 0 ), then ĥ(ξ) = e 2πix0ξ ˆf (ξ). If h(x) = e 2πixξ0 f (x), then ĥ(ξ) = ˆf (ξ ξ 0 ). f (x) 2 dx = ˆf (ξ) 2 dξ. ˆf (ξ) = 2πiξ ˆf (ξ), and f ( ˆ n)(ξ) = (2πiξ) n ˆf (ξ). (f g)(x) = f (y)g(x y) dy, then f ˆ g(ξ) = ˆf (ξ) ĝ(ξ). It is related to Laplace transform, F (s) = f (t)e st dt. 0 Discrete Fourier transform, given {f (x k )} N 1 k=0, its discrete Fourier transform {ˆf (x k )} N 1 k=0, is given by, N 1 ˆf (x k ) = f (x n )e 2πi N kn. n=0 Ming Zhong (JHU) AMS Fall / 19

5 Wavelet Transform Continuous Wavelet Transform (CWT) The CWT is a non-numerical tool that provides an overcomplete representation of a signal by adding various translation and scale parameters of the wavelet. Definition The CWT of a function f (t) at a scale a R + (i.e., a > 0) and a translation value b R is expressed by the following integral, F ω (a, b) = 1 a 1 2 f (t) ψ( t b ) dt. a Here, ψ(t) is a continuous function in both the time domain and frequency domain, which is called the mother wavelet. Ming Zhong (JHU) AMS Fall / 19

6 Wavelet Transform, cont. Furthermore, to recover f, we will do, f (t) = C 1 ψ Here, ψ is the dual function of ψ, F ω (a, b) 1 1 a 2 ψ( t b ) db da a a 2. C ψ = ˆψ(ω) ˆ ψ(ω) ω dω C ψ is the wavelet admissible constant, when a wavelet with 0 < C ψ < is called admissible, then ˆψ(0) = 0. It is also suggested, where w(t) is called a window. ψ(t) = w(t) exp(it), Ming Zhong (JHU) AMS Fall / 19

7 Wavelet Transform, cont. Furthermore, The scale factor a either dilates or compresses a signal. When a is relatively low, resulting in a more detailed resulting graph, but only lasting a short duration; when a is relatively high, resulting in less detailed graph, but lasting the entire duration. The CWT is a convolution of the input f with a set of functions generated by the mother wavelet, it can be computed using Fast Fourier Transform (FFT). Image Compression: it provides significant improvement in picture quality at higher compression ratios. Wavelet transform decomposes complex information and patterns into elementary forms, it is commonly used in acoustics processing and pattern recognition. Edge and corner detection, PDEs, transient detection, filter design, Electrocardiogram (ECG) analysis, texture analysis, etc. Ming Zhong (JHU) AMS Fall / 19

8 Principal Components Analysis (PCA) Definition Principal Component Analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linear uncorrelated variables which are called principal components. Moreover, If there are n observations with p variables, the number of distinct principal components is min(n 1, p). The principal components have the largest possible variances. IT was invented in 1901 by Karl Pearson as an analogue of the principal axis theorem in mechanics. It is also named the discrete Karhunen-Loéve transform (KLT) in signal processing. Ming Zhong (JHU) AMS Fall / 19

9 Introduction to Machine Learning Machine Learning Machine Learning (ML) is a field of artificial intelligence that uses statistical techniques to give computer systems that ability to learn from data, without being explicitly programmed. Moreover, It was first coined by Arhur Samuel in To study and construct a set of algorithms that can learn from and make predictions on data. It is closely related to computational statistics and mathematical optimization; different from data mining, which focuses more on exploring data analysis (known as unsupervised learning). Supervised, semi-supervised, active, unsupervised, and reinforcement. Ming Zhong (JHU) AMS Fall / 19

10 Image Recognition Image recognition, Automating computers to recognize people, animals, and place. Many apps, especially those on the phone: recognizing faces in pictures taken, some can identify objects, even certain locations. Uses statistics, time-frequency analysis, and the SVD. Uses dimension reduction and statistical method, a simple example of Machine Learning (aka, statistical learning). Part of the data classification scheme (pattern theory), using a prescribed set of data - supervised machine learning. unsupervised machine learning can be derived. Ming Zhong (JHU) AMS Fall / 19

11 Recognizing Dogs and Cats We will create a mathematically plausible mechanism by which we can discriminate dogs and cats, How can a figure be minimally represented? With very simple lines, one can represent a dog. Not only do we recognize a dog, but also do we recognize the breed: a duaschung. Edge detection play a critical role in image recognition. Could a computer recognize it the same way as we do? Ming Zhong (JHU) AMS Fall / 19

12 Wavelet Analysis of Images Wavelets represent an ideal way to represent multi-scale information, We will use wavelets to represent dogs and cats. Wavelets are efficient in detecting and highlighting edges in images. We will develop an algorithm by which we can train the computer to distinguish between dogs and cats, Step 1: Decompose images of dogs and cats into wavelet basis functions. The training sets for dogs and cats will be 80 images each. Step 2: From the wavelet expanded images, find the principal components associated with dogs and cats. Step 3: Design a statistical decision threshold for discrimination between dogs and cats, linear discrimination analysis (LDA) will be used. Step 4: Test the algorithm efficiency and accuracy by running it through 20 pictures of cats and 20 pictures of dogs. Ming Zhong (JHU) AMS Fall / 19

13 Step 1: Wavelet Decomposition We will need a sample of the dog pictures, We will use the data file, dogdata.mat, contains a matrix of the size will re-sized to resolution of an image. 80 columns are for 80 different dogs. This data set is for training. Similar data set for cats. load dogdata for j = 1 : 9 subplot(3, 3, j) dogj = reshape(dog(:, j), 64, 64) imshow(dogj) end Ming Zhong (JHU) AMS Fall / 19

14 Dog Pictures Note that: we do not need high-resolution images. Ming Zhong (JHU) AMS Fall / 19

15 Wavelet Decomposition, cont. We will now decompose the images using a wavelet basis using the discrete wavelet transform in MATLAB. dwt2 will perform a single-level discrete 2D wavelet transform (decomposition), with respect to a particular wavelet or particular wavelet filters. The commend: [ca, ch, cv, cd] = DWT2(X, wname ) ca: approximation coefficient matrix; ch, cv, cd: other coefficient matrices; X is an input image; wname for specific wavelet type. Large-scale features in ca; fine-scale features in the horizontal, vertical, and diagonal directions in ch, cv, cd. Ming Zhong (JHU) AMS Fall / 19

16 Haar Wavelet Decomposition The following code, figure(2) X = double(reshape(dog(:, 6), 64, 64)); [ca, ch, cv, cd] = dwt2(x, haar ); subplot(2, 2, 1), imshow(uint8(ca)); subplot(2, 2, 2), imshow(uint8(ch)); subplot(2, 2, 3), imshow(uint8(cv)); subplot(2, 2, 4), imshow(uint8(cd)); Ming Zhong (JHU) AMS Fall / 19

17 Dog Pictures The images are so dark, because they are not re-scaled to the appropriate pseudo-color scaling used by the image commands. Ming Zhong (JHU) AMS Fall / 19

18 Re-scaling, Edge Detection We will rescale the images, and combine the edge detection wavelets in the horizontal and vertical direction into a single matrix. nbcol = size(colormap(gray), 1); cod ch1 = wcodemat(ch, nbcol); cod cv1 = wcodemat(cv, nbcol); cod edge = cod ch1 + cod cv1; figure(3) subplot(2, 2, 1), imshow(uint8(cod ch1)); subplot(2, 2, 2), imshow(uint8(cod cv1)); subplot(2, 2, 3), imshow(uint8(cod edge)); subplot(2, 2, 4), imshow(reshape(dog(:, 6), 64, 64)); Ming Zhong (JHU) AMS Fall / 19

19 Dog Pictures The new images are brighter. Ming Zhong (JHU) AMS Fall / 19