Gene expression experiments. Infinite Dimensional Vector Spaces. 1. Motivation: Statistical machine learning and reproducing kernel Hilbert Spaces
|
|
- Estella Mosley
- 6 years ago
- Views:
Transcription
1 MA 751 Part 3 Gene expression experiments Infinite Dimensional Vector Spaces 1. Motivation: Statistical machine learning and reproducing kernel Hilbert Spaces Gene expression experiments Question: Gene expression - when is the DNA in a gene 1 transcribed and thus expressed (as RNA) in a cell?
2 Gene expression experiments One solution: Measure RNA levels (result of transcription) Method: Microarray or RNA Seq array Result: for each subject tissue sample =, obtain a feature vector: FÐ=Ñ œ x œ ÐB" ß á ß B25ß!!! Ñ consisting of expression levels of 25,000 genes. Can we classify tissues this way?
3 Goals: Gene expression experiments 1. Differentiate two different but similar cancers. 2. Understand genetic pathways of cancer Basic difficulties: few samples (e.g., ); high dimension (e.g., 5, ,000). Curse of dimensionality - too few samples and too many parameters (dimensions) to fit them. Tool: Support vector machine (SVM)
4 Gene expression experiments Procedure: look at feature space J in which FÐ=Ñ lives, and differentiate examples of one and the other cancer with a hyperplane:
5 Gene expression experiments Methods needed for full analysis (of SVM and other high dimensional methods): Reproducing kernel Hilbert spaces (RKHS)
6 2. Machine Learning: The role of learning theory The role of learning theory has grown a great deal in: Mathematics Statistics Computational Biology Neurosciences, e.g., theory of plasticity, workings of visual cortex
7 Source: University of Washington
8 Kernel methods Kernel methods are used widely in: Computer science, e.g., vision theory, graphics, speech synthesis
9 Kernel methods Source: T. Poggio/M
10 Kernel methods Face identification: MIT
11 Kernel methods People classification or detection: Poggio/MIT
12 Learning theory We want the theory behind such learning algorithms- 3. The learning theory problem Given an unknown function 0ÐxÑ À Ä ß learn 0ÐxÑ from a few examples, i.e., a few inputs x where 0ÐxÑ is known. Determine unknown points x. 0ÐxÑ. from knowing its value at several
13 Learning theory Example 1: x is retinal activation pattern (i.e., B 3 œ activation level of retinal neuron 3), and Cœ0ÐxÑ! if the retinal pattern is a chair; Cœ0ÐxÑ! otherwise. [Thus: want concept of a chair] Given: examples of chairs (and non-chairs): x" ßx# ßáßx8, together with proper outputs CßáßCÞ " 8 This is the information: R0 œ Ð0Ðx Ñßáß0Ðx ÑÑ " 8
14 Learning theory Goal: Give best possible estimate of the unknown function 0, i.e., try to learn the concept 0 from the examples R0. But: given pointwise information about 0 not sufficient: which is the "right" 0ÐBÑ given the data R0 below?
15 Learning theory
16 (a) Learning theory
17 (b) Learning theory [How to decide?]
18 Infinite dimensional spaces 4. Infinite dimensional vector spaces: [This material is short course in real/functional analysis; see me if you want more sources] [Notation: in infinite dimensions generally remove boldface from vectors] Let L be a vector space with inner product. Recall by definition œ m@m #.
19 Infinite dimensional spaces Recall œ œ Distance between Consider infinite collection " # " # W œö@ ß@ ß@ á LÞ " # $
20 Infinite dimensional spaces Define infinite linear combinations by: if œa 3œ" ½A -@ ½!Þ 3œ" 3 3 8Ä Ò [Definitions of span, linear independence, basis same except for infinite sums. Henceforth always allow infinite linear combinations.]
21 Infinite dimensional spaces Def. 4. All previous linear algebra definitions (e.g. spanning, linear independence, basis) extend directly to the case of infinite numbers of vectors. Example: A collection Ö@ " ß@ # ßá of vectors spans a vector space Z if every Z can be written as a (possibly infinite) linear œ-@ -@ á œ -@Þ " " # # 3 3 3œ" [Henceforth always allow infinite linear combinations.]
22 Hilbert spaces Def 5: An inner product space L is complete if any sequence ÖB L which is Cauchy, i.e., mb B m Ä! (that is, 3 3œ" 3 4 3ß4Ä it should converge) actually converges to some B L, i.e. B3 Ä BÞ [Thus if the sequence bunches up, there is something for it to converge to.] Such an inner product space Hilbert space. L that is complete is called a
23 Example of incomplete space Ex: Not all inner product spaces are Hilbert spaces since not all are complete. As an example, consider the space T œ Öall polynomials on Ò!ß "Ó. Define inner product " Ð0ß 1Ñ œ '! 0ÐBÑ1ÐBÑ.BÞ # ' " #! Then resulting norm is ll0ðbñll œ Ø0ß 0Ù œ 0 ÐBÑ.BÞ This space is not complete: consider the vectors B by the partial sums of the Taylor series for / R defined
24 Example of incomplete R R 8 œ 8œ! B 8x = a polynomial.
25 Example of incomplete space Note that if R Qthen R 8 Q 8 R 8 R 8 B B B B ll@ Qll œ ¾ ¾ œ ¾ ¾ Ÿ ½ ½ 8x 8x 8x 8x But: 8 8œ! 8œ! 8œQ " 8œQ " B " 8 " #8 " " m m œ mb m œ B.B œ Þ 8x 8x 8x ( Œ 8x #8 "! " "Î# "Î#
26 So easy to show m Example of incomplete space 8=0 8 B 8x m. Thus it easily follows that ll@ Q ll Ò!ß RßQ Ä so that the is a Cauchy sequence in L. R
27 Example of incomplete space But note that by Taylor series R 8 B R ÐBÑ / œ B / Ä! 8x RÄ 8œ! uniformly on [0,1]. Thus easy to show that B m@ R ÐBÑ / m Ò!Þ RÄ
28 So: Example of incomplete space R ÐBÑ Ä / Þ But: can show that a sequence of functions can't converge to 2 different functions. Thus there is no polynomial :ÐBÑ (i.e. something in our space T ) such ÐBÑ Ä :ÐBÑÞ R R do not converge to something in T and thus T is not complete! [Moral: intuitively, complete space is one where any convergent sequence T8 converges to an element T of the original space.]
29 Theorem 4: If F œö@ " ß@ # ß@ $ ßá is a collection of vectors that is orthonormal (i.e, unit lengths and inner product!), then it is automatically linearly independent. If F is a basis for L and is orthonormal, it is called an orthonormal basis.
30 $ Examples of Hilbert spaces Ex 2: Lœ " $ 3 is a Hilbert space (i.e., not hard to show that it's complete). Inner product is the usual one for vectors: This L is a Hilbert space. Orthonormal " " # # $ $ Ô " Ô! Ô! e" œ! à e# œ " à e$ œ! Þ Õ! Ø Õ! Ø Õ" Ø
31 Examples of Hilbert spaces Ex 3: L œ # œ second order polynomials on Ò!ß "Ó œ Ö+ + B + B À + #! " $ 3 forms a Hilbert space. Inner product: " # ' "! " # Ð: ÐBÑß : ÐBÑÑ œ : ÐBÑ: ÐBÑ.BÞ
32 Examples of Hilbert spaces Note it is not hard to show that L is complete (in fact any finite dimensional vector space is complete). Thus L is a Hilbert space. Ex 4: Note L œ œö@œð@ " ß@ # ß@ $ ßáÑl@ 3 Hilbert space, if we define the inner product Ð@ß AÑ A á A " " # # 3 3 3œ" is a
33 Examples of Hilbert spaces Length of a is m@m œ 3@ 3 œ 3 # Þ 3œ" 3œ" Thus to have well-defined lengths we add the condition that m@m to the definition of L. Then can show that L satisfies all the properties of a Hilbert space (in particular it's complete).
34 Examples of Hilbert spaces Can show that the set @ " # $ œ Ð"ß!ß!ßáÑ œ Ð!ß"ß!ßáÑ œð!ß!ß"ßáñ ã is certainly orthonormal, and it spans orthonormal basis for L. L, so it is an
Infinite Dimensional Vector Spaces. 1. Motivation: Statistical machine learning and reproducing kernel Hilbert Spaces
MA 751 Part 3 Infinite Dimensional Vector Spaces 1. Motivation: Statistical machine learning and reproducing kernel Hilbert Spaces Microarray experiment: Question: Gene expression - when is the DNA in
More informationSVM example: cancer classification Support Vector Machines
SVM example: cancer classification Support Vector Machines 1. Cancer genomics: TCGA The cancer genome atlas (TCGA) will provide high-quality cancer data for large scale analysis by many groups: SVM example:
More informationSolving SVM: Quadratic Programming
MA 751 Part 7 Solving SVM: Quadratic Programming 1. Quadratic programming (QP): Introducing Lagrange multipliers α 4 and. 4 (can be justified in QP for inequality as well as equality constraints) we define
More informationStatistical machine learning and kernel methods. Primary references: John Shawe-Taylor and Nello Cristianini, Kernel Methods for Pattern Analysis
Part 5 (MA 751) Statistical machine learning and kernel methods Primary references: John Shawe-Taylor and Nello Cristianini, Kernel Methods for Pattern Analysis Christopher Burges, A tutorial on support
More informationInner Product Spaces
Inner Product Spaces In 8 X, we defined an inner product? @? @?@ ÞÞÞ? 8@ 8. Another notation sometimes used is? @? ß@. The inner product in 8 has several important properties ( see Theorem, p. 33) that
More informationDr. H. Joseph Straight SUNY Fredonia Smokin' Joe's Catalog of Groups: Direct Products and Semi-direct Products
Dr. H. Joseph Straight SUNY Fredonia Smokin' Joe's Catalog of Groups: Direct Products and Semi-direct Products One of the fundamental problems in group theory is to catalog all the groups of some given
More information8œ! This theorem is justified by repeating the process developed for a Taylor polynomial an infinite number of times.
Taylor and Maclaurin Series We can use the same process we used to find a Taylor or Maclaurin polynomial to find a power series for a particular function as long as the function has infinitely many derivatives.
More informationSystems of Equations 1. Systems of Linear Equations
Lecture 1 Systems of Equations 1. Systems of Linear Equations [We will see examples of how linear equations arise here, and how they are solved:] Example 1: In a lab experiment, a researcher wants to provide
More informationProbability basics. Probability and Measure Theory. Probability = mathematical analysis
MA 751 Probability basics Probability and Measure Theory 1. Basics of probability 2 aspects of probability: Probability = mathematical analysis Probability = common sense Probability basics A set-up of
More informationPART I. Multiple choice. 1. Find the slope of the line shown here. 2. Find the slope of the line with equation $ÐB CÑœ(B &.
Math 1301 - College Algebra Final Exam Review Sheet Version X This review, while fairly comprehensive, should not be the only material used to study for the final exam. It should not be considered a preview
More informationInner Product Spaces. Another notation sometimes used is X.?
Inner Product Spaces X In 8 we have an inner product? @? @?@ ÞÞÞ? 8@ 8Þ Another notation sometimes used is X? ß@? @? @? @ ÞÞÞ? @ 8 8 The inner product in?@ ß in 8 has several essential properties ( see
More informationE.G., data mining, prediction. Function approximation problem: How >9 best estimate the function 0 from partial information (examples)
Can Neural Network and Statistical Learning Theory be Formulated in terms of Continuous Complexity Theory? 1. Neural Networks and Statistical Learning Theory Many areas of mathematics, statistics, and
More informationB œ c " " ã B œ c 8 8. such that substituting these values for the B 3 's will make all the equations true
System of Linear Equations variables Ð unknowns Ñ B" ß B# ß ÞÞÞ ß B8 Æ Æ Æ + B + B ÞÞÞ + B œ, "" " "# # "8 8 " + B + B ÞÞÞ + B œ, #" " ## # #8 8 # ã + B + B ÞÞÞ + B œ, 3" " 3# # 38 8 3 ã + 7" B" + 7# B#
More informationHere are proofs for some of the results about diagonalization that were presented without proof in class.
Suppose E is an 8 8 matrix. In what follows, terms like eigenvectors, eigenvalues, and eigenspaces all refer to the matrix E. Here are proofs for some of the results about diagonalization that were presented
More information: œ Ö: =? À =ß> real numbers. œ the previous plane with each point translated by : Ðfor example,! is translated to :)
â SpanÖ?ß@ œ Ö =? > @ À =ß> real numbers : SpanÖ?ß@ œ Ö: =? > @ À =ß> real numbers œ the previous plane with each point translated by : Ðfor example, is translated to :) á In general: Adding a vector :
More informationProofs Involving Quantifiers. Proof Let B be an arbitrary member Proof Somehow show that there is a value
Proofs Involving Quantifiers For a given universe Y : Theorem ÐaBÑ T ÐBÑ Theorem ÐbBÑ T ÐBÑ Proof Let B be an arbitrary member Proof Somehow show that there is a value of Y. Call it B œ +, say Þ ÐYou can
More informationChapter 9. Support Vector Machine. Yongdai Kim Seoul National University
Chapter 9. Support Vector Machine Yongdai Kim Seoul National University 1. Introduction Support Vector Machine (SVM) is a classification method developed by Vapnik (1996). It is thought that SVM improved
More informationMath 131 Exam 4 (Final Exam) F04M
Math 3 Exam 4 (Final Exam) F04M3.4. Name ID Number The exam consists of 8 multiple choice questions (5 points each) and 0 true/false questions ( point each), for a total of 00 points. Mark the correct
More informationManifold Regularization
Manifold Regularization Vikas Sindhwani Department of Computer Science University of Chicago Joint Work with Mikhail Belkin and Partha Niyogi TTI-C Talk September 14, 24 p.1 The Problem of Learning is
More informationÆ Å not every column in E is a pivot column (so EB œ! has at least one free variable) Æ Å E has linearly dependent columns
ßÞÞÞß @ is linearly independent if B" @" B# @# ÞÞÞ B: @: œ! has only the trivial solution EB œ! has only the trivial solution Ðwhere E œ Ò@ @ ÞÞÞ@ ÓÑ every column in E is a pivot column E has linearly
More informationNotes on the Unique Extension Theorem. Recall that we were interested in defining a general measure of a size of a set on Ò!ß "Ó.
1. More on measures: Notes on the Unique Extension Theorem Recall that we were interested in defining a general measure of a size of a set on Ò!ß "Ó. Defined this measure T. Defined TÐ ( +ß, ) Ñ œ, +Þ
More informationSupport Vector Machines
Wien, June, 2010 Paul Hofmarcher, Stefan Theussl, WU Wien Hofmarcher/Theussl SVM 1/21 Linear Separable Separating Hyperplanes Non-Linear Separable Soft-Margin Hyperplanes Hofmarcher/Theussl SVM 2/21 (SVM)
More informationFramework for functional tree simulation applied to 'golden delicious' apple trees
Purdue University Purdue e-pubs Open Access Theses Theses and Dissertations Spring 2015 Framework for functional tree simulation applied to 'golden delicious' apple trees Marek Fiser Purdue University
More informationLA PRISE DE CALAIS. çoys, çoys, har - dis. çoys, dis. tons, mantz, tons, Gas. c est. à ce. C est à ce. coup, c est à ce
> ƒ? @ Z [ \ _ ' µ `. l 1 2 3 z Æ Ñ 6 = Ð l sl (~131 1606) rn % & +, l r s s, r 7 nr ss r r s s s, r s, r! " # $ s s ( ) r * s, / 0 s, r 4 r r 9;: < 10 r mnz, rz, r ns, 1 s ; j;k ns, q r s { } ~ l r mnz,
More informationQUALIFYING EXAMINATION II IN MATHEMATICAL STATISTICS SATURDAY, MAY 9, Examiners: Drs. K. M. Ramachandran and G. S. Ladde
QUALIFYING EXAMINATION II IN MATHEMATICAL STATISTICS SATURDAY, MAY 9, 2015 Examiners: Drs. K. M. Ramachandran and G. S. Ladde INSTRUCTIONS: a. The quality is more important than the quantity. b. Attempt
More informationIntroduction to Diagonalization
Introduction to Diagonalization For a square matrix E, a process called diagonalization can sometimes give us more insight into how the transformation B ÈE B works. The insight has a strong geometric flavor,
More informationMAP METHODS FOR MACHINE LEARNING. Mark A. Kon, Boston University Leszek Plaskota, Warsaw University, Warsaw Andrzej Przybyszewski, McGill University
MAP METHODS FOR MACHINE LEARNING Mark A. Kon, Boston University Leszek Plaskota, Warsaw University, Warsaw Andrzej Przybyszewski, McGill University Example: Control System 1. Paradigm: Machine learning
More information7. Random variables as observables. Definition 7. A random variable (function) X on a probability measure space is called an observable
7. Random variables as observables Definition 7. A random variable (function) X on a probability measure space is called an observable Note all functions are assumed to be measurable henceforth. Note that
More informationExample Let VœÖÐBßCÑ À b-, CœB - ( see the example above). Explain why
Definition If V is a relation from E to F, then a) the domain of V œ dom ÐVÑ œ Ö+ E À b, F such that Ð+ß,Ñ V b) the range of V œ ran( VÑ œ Ö, F À b+ E such that Ð+ß,Ñ V " c) the inverse relation of V œ
More informationCellular Automaton Growth on # : Theorems, Examples, and Problems
Cellular Automaton Growth on : Theorems, Examples, and Problems (Excerpt from Advances in Applied Mathematics) Exactly 1 Solidification We will study the evolution starting from a single occupied cell
More informationETIKA V PROFESII PSYCHOLÓGA
P r a ž s k á v y s o k á š k o l a p s y c h o s o c i á l n í c h s t u d i í ETIKA V PROFESII PSYCHOLÓGA N a t á l i a S l o b o d n í k o v á v e d ú c i p r á c e : P h D r. M a r t i n S t r o u
More informationFull file at 2CT IMPULSE, IMPULSE RESPONSE, AND CONVOLUTION CHAPTER 2CT
2CT.2 UNIT IMPULSE 2CT.2. CHAPTER 2CT 2CT.2.2 (d) We can see from plot (c) that the limit?7 Ä!, B :?7 9 œb 9$ 9. This result is consistent with the sampling property: B $ œb $ 9 9 9 9 2CT.2.3 # # $ sin
More informationMonic Adjuncts of Quadratics (Developed, Composed & Typeset by: J B Barksdale Jr / )
Monic Adjuncts of Quadratics (eveloped, Composed & Typeset by: J B Barksdale Jr / 04 18 15) Article 00.: Introduction. Given a univariant, Quadratic polynomial with, say, real number coefficients, (0.1)
More informationDEPARTMENT OF MATHEMATICS AND STATISTICS QUALIFYING EXAMINATION II IN MATHEMATICAL STATISTICS SATURDAY, SEPTEMBER 24, 2016
DEPARTMENT OF MATHEMATICS AND STATISTICS QUALIFYING EXAMINATION II IN MATHEMATICAL STATISTICS SATURDAY, SEPTEMBER 24, 2016 Examiners: Drs. K. M. Ramachandran and G. S. Ladde INSTRUCTIONS: a. The quality
More informationCIS 520: Machine Learning Oct 09, Kernel Methods
CIS 520: Machine Learning Oct 09, 207 Kernel Methods Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture They may or may not cover all the material discussed
More informationSIMULTANEOUS CONFIDENCE BANDS FOR THE PTH PERCENTILE AND THE MEAN LIFETIME IN EXPONENTIAL AND WEIBULL REGRESSION MODELS. Ping Sa and S.J.
SIMULTANEOUS CONFIDENCE BANDS FOR THE PTH PERCENTILE AND THE MEAN LIFETIME IN EXPONENTIAL AND WEIBULL REGRESSION MODELS " # Ping Sa and S.J. Lee " Dept. of Mathematics and Statistics, U. of North Florida,
More informationEngineering Mathematics (E35 317) Final Exam December 18, 2007
Engineering Mathematics (E35 317) Final Exam December 18, 2007 This exam contains 18 multile-choice roblems orth to oints each, five short-anser roblems orth one oint each, and nine true-false roblems
More informationA short introduction to supervised learning, with applications to cancer pathway analysis Dr. Christina Leslie
A short introduction to supervised learning, with applications to cancer pathway analysis Dr. Christina Leslie Computational Biology Program Memorial Sloan-Kettering Cancer Center http://cbio.mskcc.org/leslielab
More informationReproducing Kernel Hilbert Spaces Class 03, 15 February 2006 Andrea Caponnetto
Reproducing Kernel Hilbert Spaces 9.520 Class 03, 15 February 2006 Andrea Caponnetto About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More informationAutomatic Control III (Reglerteknik III) fall Nonlinear systems, Part 3
Automatic Control III (Reglerteknik III) fall 20 4. Nonlinear systems, Part 3 (Chapter 4) Hans Norlander Systems and Control Department of Information Technology Uppsala University OSCILLATIONS AND DESCRIBING
More informationSIMULATION - PROBLEM SET 1
SIMULATION - PROBLEM SET 1 " if! Ÿ B Ÿ 1. The random variable X has probability density function 0ÐBÑ œ " $ if Ÿ B Ÿ.! otherwise Using the inverse transform method of simulation, find the random observation
More informationHilbert Spaces: Infinite-Dimensional Vector Spaces
Hilbert Spaces: Infinite-Dimensional Vector Spaces PHYS 500 - Southern Illinois University October 27, 2016 PHYS 500 - Southern Illinois University Hilbert Spaces: Infinite-Dimensional Vector Spaces October
More information1 Mathematical preliminaries
1 Mathematical preliminaries The mathematical language of quantum mechanics is that of vector spaces and linear algebra. In this preliminary section, we will collect the various definitions and mathematical
More information1. Classify each number. Choose all correct answers. b. È # : (i) natural number (ii) integer (iii) rational number (iv) real number
Review for Placement Test To ypass Math 1301 College Algebra Department of Computer and Mathematical Sciences University of Houston-Downtown Revised: Fall 2009 PLEASE READ THE FOLLOWING CAREFULLY: 1. The
More information! " # $! % & '! , ) ( + - (. ) ( ) * + / 0 1 2 3 0 / 4 5 / 6 0 ; 8 7 < = 7 > 8 7 8 9 : Œ Š ž P P h ˆ Š ˆ Œ ˆ Š ˆ Ž Ž Ý Ü Ý Ü Ý Ž Ý ê ç è ± ¹ ¼ ¹ ä ± ¹ w ç ¹ è ¼ è Œ ¹ ± ¹ è ¹ è ä ç w ¹ ã ¼ ¹ ä ¹ ¼ ¹ ±
More informationMath 131, Exam 2 Solutions, Fall 2010
Math 131, Exam 2 Solutions, Fall 2010 Part I consists of 14 multiple choice questions (orth 5 points each) and 5 true/false questions (orth 1 point each), for a total of 75 points. Mark the correct anser
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines Shivani Agarwal Support Vector Machines (SVMs) Algorithm for learning linear classifiers Motivated by idea of maximizing margin Efficient extension to non-linear
More informationFunction Space and Convergence Types
Function Space and Convergence Types PHYS 500 - Southern Illinois University November 1, 2016 PHYS 500 - Southern Illinois University Function Space and Convergence Types November 1, 2016 1 / 7 Recall
More informationLecture 16: Modern Classification (I) - Separating Hyperplanes
Lecture 16: Modern Classification (I) - Separating Hyperplanes Outline 1 2 Separating Hyperplane Binary SVM for Separable Case Bayes Rule for Binary Problems Consider the simplest case: two classes are
More informationNECESSARY AND SUFFICIENT CONDITION FOR ASYMPTOTIC NORMALITY OF STANDARDIZED SAMPLE MEANS
NECESSARY AND SUFFICIENT CONDITION FOR ASYMPTOTIC NORMALITY OF STANDARDIZED SAMPLE MEANS BY RAJESHWARI MAJUMDAR* AND SUMAN MAJUMDAR University of Connecticut and University of Connecticut The double sequence
More informationKernel Methods. Barnabás Póczos
Kernel Methods Barnabás Póczos Outline Quick Introduction Feature space Perceptron in the feature space Kernels Mercer s theorem Finite domain Arbitrary domain Kernel families Constructing new kernels
More informationAnalysis of Spectral Kernel Design based Semi-supervised Learning
Analysis of Spectral Kernel Design based Semi-supervised Learning Tong Zhang IBM T. J. Watson Research Center Yorktown Heights, NY 10598 Rie Kubota Ando IBM T. J. Watson Research Center Yorktown Heights,
More informationDay 4: Classification, support vector machines
Day 4: Classification, support vector machines Introduction to Machine Learning Summer School June 18, 2018 - June 29, 2018, Chicago Instructor: Suriya Gunasekar, TTI Chicago 21 June 2018 Topics so far
More informationApplied Machine Learning Annalisa Marsico
Applied Machine Learning Annalisa Marsico OWL RNA Bionformatics group Max Planck Institute for Molecular Genetics Free University of Berlin 29 April, SoSe 2015 Support Vector Machines (SVMs) 1. One of
More informationMIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications. Class 04: Features and Kernels. Lorenzo Rosasco
MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 04: Features and Kernels Lorenzo Rosasco Linear functions Let H lin be the space of linear functions f(x) = w x. f w is one
More informationChapter 2. Linear Algebra. rather simple and learning them will eventually allow us to explain the strange results of
Chapter 2 Linear Algebra In this chapter, we study the formal structure that provides the background for quantum mechanics. The basic ideas of the mathematical machinery, linear algebra, are rather simple
More informationVectors. Teaching Learning Point. Ç, where OP. l m n
Vectors 9 Teaching Learning Point l A quantity that has magnitude as well as direction is called is called a vector. l A directed line segment represents a vector and is denoted y AB Å or a Æ. l Position
More informationMATH 1301 (College Algebra) - Final Exam Review
MATH 1301 (College Algebra) - Final Exam Review This review is comprehensive but should not be the only material used to study for the final exam. It should not be considered a preview of the final exam.
More informationAN IDENTIFICATION ALGORITHM FOR ARMAX SYSTEMS
AN IDENTIFICATION ALGORITHM FOR ARMAX SYSTEMS First the X, then the AR, finally the MA Jan C. Willems, K.U. Leuven Workshop on Observation and Estimation Ben Gurion University, July 3, 2004 p./2 Joint
More informationThe Hahn-Banach and Radon-Nikodym Theorems
The Hahn-Banach and Radon-Nikodym Theorems 1. Signed measures Definition 1. Given a set Hand a 5-field of sets Y, we define a set function. À Y Ä to be a signed measure if it has all the properties of
More informationAn Introduction to Optimal Control Applied to Disease Models
An Introduction to Optimal Control Applied to Disease Models Suzanne Lenhart University of Tennessee, Knoxville Departments of Mathematics Lecture1 p.1/37 Example Number of cancer cells at time (exponential
More informationB-9= B.Bà B 68 B.Bß B/.Bß B=38 B.B >+8 ab b.bß ' 68aB b.bà
8.1 Integration y Parts.@ a.? a Consider. a? a @ a œ? a @ a Þ....@ a..? a We can write this as? a œ a? a @ a@ a Þ... If we integrate oth sides, we otain.@ a a.?. œ a? a @ a..? a @ a. or...? a.@ œ? a @
More informationImplicit in the definition of orthogonal arrays is a projection propertyþ
Projection properties Implicit in the definition of orthogonal arrays is a projection propertyþ factor screening effect ( factor) sparsity A design is said to be of projectivity : if in every subset of
More informationOptimal data-independent point locations for RBF interpolation
Optimal dataindependent point locations for RF interpolation S De Marchi, R Schaback and H Wendland Università di Verona (Italy), Universität Göttingen (Germany) Metodi di Approssimazione: lezione dell
More informationHuman genome contains ~30K genes Not all genes are expressed at same time
! " $#% & & '() Eric P. Xing, Michael I. Jordan, & Richard M. Karp @ Division of Computer Science, UC Bereley Presented by Degui Zhi @ UCSD Apr 3 th 22 1 *,+ -/. 213.. 1 46587)+ 29 2: 4;+ =@? 9 + A)B
More informationT i t l e o f t h e w o r k : L a M a r e a Y o k o h a m a. A r t i s t : M a r i a n o P e n s o t t i ( P l a y w r i g h t, D i r e c t o r )
v e r. E N G O u t l i n e T i t l e o f t h e w o r k : L a M a r e a Y o k o h a m a A r t i s t : M a r i a n o P e n s o t t i ( P l a y w r i g h t, D i r e c t o r ) C o n t e n t s : T h i s w o
More informationKernels for Multi task Learning
Kernels for Multi task Learning Charles A Micchelli Department of Mathematics and Statistics State University of New York, The University at Albany 1400 Washington Avenue, Albany, NY, 12222, USA Massimiliano
More informationSection 1.3 Functions and Their Graphs 19
23. 0 1 2 24. 0 1 2 y 0 1 0 y 1 0 0 Section 1.3 Functions and Their Graphs 19 3, Ÿ 1, 0 25. y œ 26. y œ œ 2, 1 œ, 0 Ÿ " 27. (a) Line through a!ß! band a"ß " b: y œ Line through a"ß " band aß! b: y œ 2,
More informationChapter Seven The Quantum Mechanical Simple Harmonic Oscillator
Capter Seven Te Quantum Mecanical Simple Harmonic Oscillator Introduction Te potential energy function for a classical, simple armonic oscillator is given by ZÐBÑ œ 5B were 5 is te spring constant. Suc
More informationReproducing Kernel Hilbert Spaces
Reproducing Kernel Hilbert Spaces Lorenzo Rosasco 9.520 Class 03 February 9, 2011 About this class Goal In this class we continue our journey in the world of RKHS. We discuss the Mercer theorem which gives
More informationLecture 4 February 2
4-1 EECS 281B / STAT 241B: Advanced Topics in Statistical Learning Spring 29 Lecture 4 February 2 Lecturer: Martin Wainwright Scribe: Luqman Hodgkinson Note: These lecture notes are still rough, and have
More informationSuggestions - Problem Set (a) Show the discriminant condition (1) takes the form. ln ln, # # R R
Suggetion - Problem Set 3 4.2 (a) Show the dicriminant condition (1) take the form x D Ð.. Ñ. D.. D. ln ln, a deired. We then replace the quantitie. 3ß D3 by their etimate to get the proper form for thi
More informationEngineering Mathematics (E35 317) Exam 3 November 7, 2007
Engineering Mathematics (E35 317) Exam 3 November 7, 2007 This exam contains four multile-choice roblems worth two oints each, twelve true-false roblems worth one oint each, and four free-resonse roblems
More informationThe Rational Numbers
The Rational Numbers Fields The system of integers that e formally defined is an improvement algebraically on the hole number system = (e can subtract in ) But still has some serious deficiencies: for
More informationPlanning for Reactive Behaviors in Hide and Seek
University of Pennsylvania ScholarlyCommons Center for Human Modeling and Simulation Department of Computer & Information Science May 1995 Planning for Reactive Behaviors in Hide and Seek Michael B. Moore
More informationEquivalent Sets. It is intuitively clear that for finite sets E F iff Eand Fhave the same number of elementsþ
Equivalent Sets Definition Let Eß F be sets. We say that E is equivalent to F iff there exists a bijection 0ÀEÄF. If Eis equivalent to F, we write E FÐor E For EµFor something similar: the notation varies
More informationCHAPTER I INTRODUCTION
CHAPTER I INTRODUCTION Preliminaries. Perturbation theory is the study of the effects of small disturbances in the mathematical model of a physical system. The model could be expressed as an algebraic
More informationCHAPTER V MULTIPLE SCALES..? # w. 5?œ% 0 a?ß?ß%.?.? # %?œ!.>#.>
CHAPTER V MULTIPLE SCALES This chapter and the next concern initial value prolems of oscillatory type on long intervals of time. Until Chapter VII e ill study autonomous oscillatory second order initial
More informationIE 400 Principles of Engineering Management. Graphical Solution of 2-variable LP Problems
IE 400 Principles of Engineering Management Graphical Solution of 2-variable LP Problems Graphical Solution of 2-variable LP Problems Ex 1.a) max x 1 + 3 x 2 s.t. x 1 + x 2 6 - x 1 + 2x 2 8 x 1, x 2 0,
More informationDefinition Suppose M is a collection (set) of sets. M is called inductive if
Definition Suppose M is a collection (set) of sets. M is called inductive if a) g M, and b) if B Mß then B MÞ Then we ask: are there any inductive sets? Informally, it certainly looks like there are. For
More informationThe Learning Problem and Regularization Class 03, 11 February 2004 Tomaso Poggio and Sayan Mukherjee
The Learning Problem and Regularization 9.520 Class 03, 11 February 2004 Tomaso Poggio and Sayan Mukherjee About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing
More informationThe Spotted Owls 5 " 5. = 5 " œ!þ")45 Ð'!% leave nest, 30% of those succeed) + 5 " œ!þ("= 5!Þ*%+ 5 ( adults live about 20 yrs)
The Spotted Owls Be sure to read the example at the introduction to Chapter in the textbook. It gives more general background information for this example. The example leads to a dynamical system which
More informationLecture 7: Kernels for Classification and Regression
Lecture 7: Kernels for Classification and Regression CS 194-10, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 15, 2011 Outline Outline A linear regression problem Linear auto-regressive
More informationLinear Algebra in Computer Vision. Lecture2: Basic Linear Algebra & Probability. Vector. Vector Operations
Linear Algebra in Computer Vision CSED441:Introduction to Computer Vision (2017F Lecture2: Basic Linear Algebra & Probability Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Mathematics in vector space Linear
More informationOn Least Squares Linear Regression Without Second Moment
On Least Squares Linear Regression Without Second Moment BY RAJESHWARI MAJUMDAR University of Connecticut If \ and ] are real valued random variables such that the first moments of \, ], and \] exist and
More information2. Let 0 be as in problem 1. Find the Fourier coefficient,&. (In other words, find the constant,& the Fourier series for 0.)
Engineering Mathematics (ESE 317) Exam 4 April 23, 200 This exam contains seven multiple-choice problems worth two points each, 11 true-false problems worth one point each, and some free-response problems
More informationMAT 211, Spring 2015, Introduction to Linear Algebra.
MAT 211, Spring 2015, Introduction to Linear Algebra. Lecture 04, 53103: MWF 10-10:53 AM. Location: Library W4535 Contact: mtehrani@scgp.stonybrook.edu Final Exam: Monday 5/18/15 8:00 AM-10:45 AM The aim
More informationLecture 14 : Online Learning, Stochastic Gradient Descent, Perceptron
CS446: Machine Learning, Fall 2017 Lecture 14 : Online Learning, Stochastic Gradient Descent, Perceptron Lecturer: Sanmi Koyejo Scribe: Ke Wang, Oct. 24th, 2017 Agenda Recap: SVM and Hinge loss, Representer
More informationStatistical learning theory, Support vector machines, and Bioinformatics
1 Statistical learning theory, Support vector machines, and Bioinformatics Jean-Philippe.Vert@mines.org Ecole des Mines de Paris Computational Biology group ENS Paris, november 25, 2003. 2 Overview 1.
More informationSupport'Vector'Machines. Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan
Support'Vector'Machines Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan kasthuri.kannan@nyumc.org Overview Support Vector Machines for Classification Linear Discrimination Nonlinear Discrimination
More informationJuan Juan Salon. EH National Bank. Sandwich Shop Nail Design. OSKA Beverly. Chase Bank. Marina Rinaldi. Orogold. Mariposa.
( ) X é X é Q Ó / 8 ( ) Q / ( ) ( ) : ( ) : 44-3-8999 433 4 z 78-19 941, #115 Z 385-194 77-51 76-51 74-7777, 75-5 47-55 74-8141 74-5115 78-3344 73-3 14 81-4 86-784 78-33 551-888 j 48-4 61-35 z/ zz / 138
More informationProduct Measures and Fubini's Theorem
Product Measures and Fubini's Theorem 1. Product Measures Recall: Borel sets U in are generated by open sets. They are also generated by rectangles VœN " á N hich are products of intervals NÞ 3 Let V be
More informationJeff Howbert Introduction to Machine Learning Winter
Classification / Regression Support Vector Machines Jeff Howbert Introduction to Machine Learning Winter 2012 1 Topics SVM classifiers for linearly separable classes SVM classifiers for non-linearly separable
More informationLesson 6.6 Radical Equations
Lesson 6.6 Radical Equations Activity 1 Radical Equations 1. a. To solve a radical equation, we: (1) isolate the radical, then () square both sides. Be careful when squaring a binomial! For example, #
More informationprobability of k samples out of J fall in R.
Nonparametric Techniques for Density Estimation (DHS Ch. 4) n Introduction n Estimation Procedure n Parzen Window Estimation n Parzen Window Example n K n -Nearest Neighbor Estimation Introduction Suppose
More informationTOTAL DIFFERENTIALS. In Section B-4, we observed that.c is a pretty good approximation for? C, which is the increment of
TOTAL DIFFERENTIALS The differential was introduced in Section -4. Recall that the differential of a function œ0ðñis w. œ 0 ÐÑ.Þ Here. is the differential with respect to the independent variable and is
More informationCS 231A Section 1: Linear Algebra & Probability Review
CS 231A Section 1: Linear Algebra & Probability Review 1 Topics Support Vector Machines Boosting Viola-Jones face detector Linear Algebra Review Notation Operations & Properties Matrix Calculus Probability
More informationSupport Vector Machines. Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar
Data Mining Support Vector Machines Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar 02/03/2018 Introduction to Data Mining 1 Support Vector Machines Find a linear hyperplane
More informationLecture 10: Support Vector Machine and Large Margin Classifier
Lecture 10: Support Vector Machine and Large Margin Classifier Applied Multivariate Analysis Math 570, Fall 2014 Xingye Qiao Department of Mathematical Sciences Binghamton University E-mail: qiao@math.binghamton.edu
More informationMonic Adjuncts of Quadratics (Developed, Composed & Typeset by: J B Barksdale Jr / )
Monic Adjuncts of Quadratics (eveloped, Composed & Typeset by: J B Barksdale Jr / 04 18 15) Article 00.: Introduction. Given a univariant, Quadratic polynomial with, say, real number coefficients, (0.1)
More information