Linear Associator Linear Layer
|
|
- Norah Gibson
- 5 years ago
- Views:
Transcription
1 Hebbia Learig opic 6 Note: lecture otes by Michael Negevitsky (uiversity of asmaia) Bob Keller (Harvey Mudd College CA) ad Marti Haga (Uiversity of Colorado) are used Mai idea: learig based o associatio betwee euros he mai property of a eural etwork is a ability to lear from its eviromet ad to improve its performace through learig. So far we have cosidered supervised or active learig learig with a exteral teacher or a supervisor who presets a traiig set to the etwork. But aother type of learig also exists: usupervised learig.. I cotrast to supervised learig usupervised or self-orgaised learig does ot require a exteral teacher. Durig the traiig sessio the eural etwork receives a umber of differet iput patters discovers sigificat features i these patters ad lears how to classify iput data ito appropriate categories. Usupervised learig teds to follow the euro-biological orgaisatio of the brai. Usupervised learig algorithms aim to lear rapidly ad ca be used i real-time. Hebbia learig I 99 Doald Hebb proposed oe of the key ideas i biological learig commoly kow as Hebb s Law. Hebb s Law states that if euro i is ear eough to excite euro j ad repeatedly participates i its activatio the syaptic coectio betwee these two euros is stregtheed ad euro j becomes more sesitive to stimuli from euro i. Hebb s Postulate Whe a axo of cell A is ear eough to excite a cell B ad repeatedly or persistetly takes part i firig it some growth process or metabolic chage takes place i oe or both cells such that A s efficiecy as oe of the cells firig B is icreased. D. O. Hebb 99 Dedrites B Syapse Cell Body Axo A Iputs R Liear Associator p R x W Liear Layer S x R a S a S x S x a = pureli (Wp) = Wp raiig Set: R a = i w p j j = { p t } { p t } { p Q t } Q Hebb Rule w ew = Simplified Form: Supervised Form: Matrix Form: wold + α f i ( a iq )g j ( p jq ) Postsyaptic Sigal w ew = w old+ αa iq p jq w ew = w old + t iq p jq W ew = W old + t q p q Presyaptic Sigal
2 Matrix Form: Batch Operatio Q W t p t p = t Q p Q = t q p q q = p W = p t t t Q = P p Q P = p p p Q = t t t Q (Zero Iitial Weights) Hebb s Law ca be represeted i the form of two rules:. If two euros o either side of a coectio are activated sychroously the the weight of that coectio is icreased.. If two euros o either side of a coectio are activated asychroously the the weight of that coectio is decreased (added later) Hebb s Law provides the basis for learig without a teacher. Learig here is a local pheomeo occurrig without feedback from the eviromet. I p u t S i g a l s Hebbia learig i a eural etwork i j O u t p u t S i g a l s Hebbia learig implies that weights ca oly icrease. o resolve this problem we might impose a limit o the growth of syaptic weights. It ca be doe by itroducig a o-liear forgettig factor ito Hebb s Law: w = α y j xi ϕ y j w where ϕ is the forgettig factor. Forgettig factor usually falls i the iterval betwee ad typically betwee. ad. to allow oly a little forgettig while limitig the weight growth. Simple Associative Network Iputs p w Hard Limit Neuro a Σ b = -.5 a = hardlim (wp + b) a = hardlim( wp + b) = hardlim( wp.5 ) stimulus respose p = a = o stimulus o respose Shape Fruit Network Baaa? Baaa Associator Smell Sight of baaa p Smell of baaa p Ucoditioed Stimulus Iputs Hard Limit Neuro w = Σ w = p shape detected = p = shape ot detected b = -.5 a = hardlim (w p + w p + b) Coditioed Stimulus smell detected smell ot detected a Baaa?
3 Usupervised Hebb Rule w q ( ) = w ( q ) + αa i ( q)p j ( q) Vector Form: W( q) = W ( q ) + αa( q )p ( q) raiig Sequece: p()p () p () Q Baaa Recogitio Example Iitial Weights: w = w( ) = raiig Sequece: { p ( ) = p( ) = } { p ( ) = p( ) = } α = wq () = wq ( ) + aq ( )pq ( ) First Iteratio (sight fails): a( ) = hardlim( w p ( ) + w( )p( ).5) = hardlim( +.5) = (o respose) w( ) = w ( ) + a( )p( ) = + = Example Secod Iteratio (sight works): a( ) = hardlim( w p ( ) + w( )p( ).5) = hardlim( +.5) = (baaa) w( ) = w ( ) + a( )p( ) = + = hird Iteratio (sight fails): a( ) = hardlim( w p ( ) + w ( )p( ).5) = hardlim( +.5) = (baaa) w( ) = w( ) + a( )p( ) = + = Baaa will ow be detected if either sesor works. Problems with Hebb Rule Weights ca become arbitrarily large here is o mechaism for weights to decrease Hebb Rule with Decay W ( q) = W ( q ) + αa( q)p( q) γw( q ) W ( q ) = ( γ )W ( q ) + αa( q)p( q) his keeps the weight matrix from growig without boud which ca be demostrated by settig both a i ad p j to : wmax = ( γ )wmax + α a p i j wmax = ( γ )wmax + α wmax = α -- γ Example: Baaa Associator α = γ =. First Iteratio (sight fails): a( ) = hardlim( w p ( ) + w( )p( ).5) = hardlim( +.5) = (o respose) w ( ) = w ( ) + a( )p( ). w( ) = +. ( ) = Secod Iteratio (sight works): a( ) = hardlim( w p ( ) + w( )p( ).5) = hardlim( +.5) = (baaa) w() = w( ) + a( )p( ).w( ) = +.( ) =
4 Example hird Iteratio (sight fails): a( ) = hardlim( w p ( ) + w ( )p( ).5) = hardlim( +.5) = (baaa) w( ) = w( ) + a( )p( ).w( ) = +.( ) =.9 Hebb Rule 8 6 max α w = -- = ---- = γ. Hebb with Decay Problem of Hebb with Decay Associatios will decay away if stimuli are ot occasioally preseted. If a i = the w ( q) = ( γ )w ( q ) If γ = this becomes w ( q) = (.9)w ( q ) herefore the weight decays by % at each iteratio where there is o stimulus. Usig Hebb s Law we ca express the adjustmet applied to the weight w at iteratio p i the followig form: w ( p ) = F [ y j ( p ) x i ( p )] As a special case we ca represet Hebb s Law as follows: w = α y x j i where α is the learig rate parameter. his equatio is referred to as the activity product rule. Hebbia learig algorithm Step : Iitialisatio. Set iitial syaptic weights ad thresholds to small radom values say i a iterval [ ]. Step : Activatio. Compute the euro output at iteratio p y j = xi w θ j i= where is the umber of euro iputs ad θ j is the threshold value of euro j. Step : Learig. Update the weights i the etwork: w( p + ) = w( p) + w( p) where w (p)) is the weight correctio at iteratio p. he weight correctio is determied by the geeralised activity product rule: w = ϕ y j [ λ xi w ] Step : Iteratio. Icrease iteratio p by oe go back to Step. Hebbia learig examplee xample o illustrate Hebbia learig cosider a fully coected feedforward etwork with a sigle layer of five computatio euros. Each euro is represeted by a McCulloch ad Pitts model with the sig activatio fuctio. he etwork is traied o the followig set of iput vectors: X = X = X = X = X 5 =
5 Iitial ad fial states of the etwork x x x x x 5 5 Iput layer y y y y 5 y 5 Output layer x x x x x 5 5 Iput layer y y y y 5 y 5 Output layer I p u t l a y e r Iitial ad fial weight matrices O u t p u t l a y e r 5 5 I p u t l a y e r O u t p u t l a y e r A test iput vector or probe is defied as X = Whe this probe is preseted to the etwork we obtai: Y = sig = Variatios of Hebbia Learig Basic Rule: W ew W old = + t q p q Learig Rate: W ew W old = + α t q p q Smoothig: W ew W old + α t q p q γ W old ( γ )W old = = + αt q p q Delta Rule: W ew = W old + α( t q a q )p q Usupervised: W ew = W old + αa q p q
Week 1, Lecture 2. Neural Network Basics. Announcements: HW 1 Due on 10/8 Data sets for HW 1 are online Project selection 10/11. Suggested reading :
ME 537: Learig-Based Cotrol Week 1, Lecture 2 Neural Network Basics Aoucemets: HW 1 Due o 10/8 Data sets for HW 1 are olie Proect selectio 10/11 Suggested readig : NN survey paper (Zhag Chap 1, 2 ad Sectios
More informationME 539, Fall 2008: Learning-Based Control
ME 539, Fall 2008: Learig-Based Cotrol Neural Network Basics 10/1/2008 & 10/6/2008 Uiversity Orego State Neural Network Basics Questios??? Aoucemet: Homework 1 has bee posted Due Friday 10/10/08 at oo
More informationPerceptron. Inner-product scalar Perceptron. XOR problem. Gradient descent Stochastic Approximation to gradient descent 5/10/10
Perceptro Ier-product scalar Perceptro Perceptro learig rule XOR problem liear separable patters Gradiet descet Stochastic Approximatio to gradiet descet LMS Adalie 1 Ier-product et =< w, x >= w x cos(θ)
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationIntermittent demand forecasting by using Neural Network with simulated data
Proceedigs of the 011 Iteratioal Coferece o Idustrial Egieerig ad Operatios Maagemet Kuala Lumpur, Malaysia, Jauary 4, 011 Itermittet demad forecastig by usig Neural Network with simulated data Nguye Khoa
More informationJacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3
No-Parametric Techiques Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3 Parametric vs. No-Parametric Parametric Based o Fuctios (e.g Normal Distributio) Uimodal Oly oe peak Ulikely real data cofies
More informationComputing the output response of LTI Systems.
Computig the output respose of LTI Systems. By breaig or decomposig ad represetig the iput sigal to the LTI system ito terms of a liear combiatio of a set of basic sigals. Usig the superpositio property
More informationIntroduction to Signals and Systems, Part V: Lecture Summary
EEL33: Discrete-Time Sigals ad Systems Itroductio to Sigals ad Systems, Part V: Lecture Summary Itroductio to Sigals ad Systems, Part V: Lecture Summary So far we have oly looked at examples of o-recursive
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationAdaptive Resonance Theory (ART)
Adaptive Resoace Theory : Soft Computig Course Lecture 25-28, otes, slides www.myreaders.ifo/, RC Chakraborty, e-mail rcchak@gmail.com, Dec., 2 http://www.myreaders.ifo/html/soft_computig.html www.myreaders.ifo
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationChE 471 Lecture 10 Fall 2005 SAFE OPERATION OF TUBULAR (PFR) ADIABATIC REACTORS
SAFE OPERATION OF TUBULAR (PFR) ADIABATIC REACTORS I a exothermic reactio the temperature will cotiue to rise as oe moves alog a plug flow reactor util all of the limitig reactat is exhausted. Schematically
More informationPattern recognition systems Laboratory 10 Linear Classifiers and the Perceptron Algorithm
Patter recogitio systems Laboratory 10 Liear Classifiers ad the Perceptro Algorithm 1. Objectives his laboratory sessio presets the perceptro learig algorithm for the liear classifier. We will apply gradiet
More informationPixel Recurrent Neural Networks
Pixel Recurret Neural Networks Aa ro va de Oord, Nal Kalchbreer, Koray Kavukcuoglu Google DeepMid August 2016 Preseter - Neha M Example problem (completig a image) Give the first half of the image, create
More informationOPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES
OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES Peter M. Maurer Why Hashig is θ(). As i biary search, hashig assumes that keys are stored i a array which is idexed by a iteger. However, hashig attempts to bypass
More informationMultilayer perceptrons
Multilayer perceptros If traiig set is ot liearly separable, a etwork of McCulloch-Pitts uits ca give a solutio If o loop exists i etwork, called a feedforward etwork (else, recurret etwork) A two-layer
More informationNUMERICAL METHODS FOR SOLVING EQUATIONS
Mathematics Revisio Guides Numerical Methods for Solvig Equatios Page 1 of 11 M.K. HOME TUITION Mathematics Revisio Guides Level: GCSE Higher Tier NUMERICAL METHODS FOR SOLVING EQUATIONS Versio:. Date:
More informationTopics Machine learning: lecture 2. Review: the learning problem. Hypotheses and estimation. Estimation criterion cont d. Estimation criterion
.87 Machie learig: lecture Tommi S. Jaakkola MIT CSAIL tommi@csail.mit.edu Topics The learig problem hypothesis class, estimatio algorithm loss ad estimatio criterio samplig, empirical ad epected losses
More informationECE-S352 Introduction to Digital Signal Processing Lecture 3A Direct Solution of Difference Equations
ECE-S352 Itroductio to Digital Sigal Processig Lecture 3A Direct Solutio of Differece Equatios Discrete Time Systems Described by Differece Equatios Uit impulse (sample) respose h() of a DT system allows
More informationMath 312 Lecture Notes One Dimensional Maps
Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,
More informationStatistical Pattern Recognition
Statistical Patter Recogitio Classificatio: No-Parametric Modelig Hamid R. Rabiee Jafar Muhammadi Sprig 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Ageda Parametric Modelig No-Parametric Modelig
More information1.3 Convergence Theorems of Fourier Series. k k k k. N N k 1. With this in mind, we state (without proof) the convergence of Fourier series.
.3 Covergece Theorems of Fourier Series I this sectio, we preset the covergece of Fourier series. A ifiite sum is, by defiitio, a limit of partial sums, that is, a cos( kx) b si( kx) lim a cos( kx) b si(
More informationAn Introduction to Neural Networks
A Itroductio to Neural Networks Referece: B.J.A. Kröse ad P.P. va der Smagt (1994): A Itroductio to Neural Networks, Poglavja 1-5, 6.1, 6.2, 7-8. Systems modellig from data 0 B.J.A. Kröse ad P.P. va der
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationFIR Filters. Lecture #7 Chapter 5. BME 310 Biomedical Computing - J.Schesser
FIR Filters Lecture #7 Chapter 5 8 What Is this Course All About? To Gai a Appreciatio of the Various Types of Sigals ad Systems To Aalyze The Various Types of Systems To Lear the Skills ad Tools eeded
More informationENGI 4421 Confidence Intervals (Two Samples) Page 12-01
ENGI 44 Cofidece Itervals (Two Samples) Page -0 Two Sample Cofidece Iterval for a Differece i Populatio Meas [Navidi sectios 5.4-5.7; Devore chapter 9] From the cetral limit theorem, we kow that, for sufficietly
More information1 Review of Probability & Statistics
1 Review of Probability & Statistics a. I a group of 000 people, it has bee reported that there are: 61 smokers 670 over 5 960 people who imbibe (drik alcohol) 86 smokers who imbibe 90 imbibers over 5
More informationCastiel, Supernatural, Season 6, Episode 18
13 Differetial Equatios the aswer to your questio ca best be epressed as a series of partial differetial equatios... Castiel, Superatural, Seaso 6, Episode 18 A differetial equatio is a mathematical equatio
More informationCSI 2101 Discrete Structures Winter Homework Assignment #4 (100 points, weight 5%) Due: Thursday, April 5, at 1:00pm (in lecture)
CSI 101 Discrete Structures Witer 01 Prof. Lucia Moura Uiversity of Ottawa Homework Assigmet #4 (100 poits, weight %) Due: Thursday, April, at 1:00pm (i lecture) Program verificatio, Recurrece Relatios
More informationZeros of Polynomials
Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree
More informationThe Phi Power Series
The Phi Power Series I did this work i about 0 years while poderig the relatioship betwee the golde mea ad the Madelbrot set. I have fially decided to make it available from my blog at http://semresearch.wordpress.com/.
More informationADVANCED DIGITAL SIGNAL PROCESSING
ADVANCED DIGITAL SIGNAL PROCESSING PROF. S. C. CHAN (email : sccha@eee.hku.hk, Rm. CYC-702) DISCRETE-TIME SIGNALS AND SYSTEMS MULTI-DIMENSIONAL SIGNALS AND SYSTEMS RANDOM PROCESSES AND APPLICATIONS ADAPTIVE
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 11
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract We will itroduce the otio of reproducig kerels ad associated Reproducig Kerel Hilbert Spaces (RKHS). We will cosider couple
More informationTHE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.
THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of
More informationResponse Variable denoted by y it is the variable that is to be predicted measure of the outcome of an experiment also called the dependent variable
Statistics Chapter 4 Correlatio ad Regressio If we have two (or more) variables we are usually iterested i the relatioship betwee the variables. Associatio betwee Variables Two variables are associated
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More informationCS:3330 (Prof. Pemmaraju ): Assignment #1 Solutions. (b) For n = 3, we will have 3 men and 3 women with preferences as follows: m 1 : w 3 > w 1 > w 2
Shiyao Wag CS:3330 (Prof. Pemmaraju ): Assigmet #1 Solutios Problem 1 (a) Cosider iput with me m 1, m,..., m ad wome w 1, w,..., w with the followig prefereces: All me have the same prefereces for wome:
More informationOrthogonal Gaussian Filters for Signal Processing
Orthogoal Gaussia Filters for Sigal Processig Mark Mackezie ad Kiet Tieu Mechaical Egieerig Uiversity of Wollogog.S.W. Australia Abstract A Gaussia filter usig the Hermite orthoormal series of fuctios
More informationPrinciple Of Superposition
ecture 5: PREIMINRY CONCEP O RUCUR NYI Priciple Of uperpositio Mathematically, the priciple of superpositio is stated as ( a ) G( a ) G( ) G a a or for a liear structural system, the respose at a give
More informationRecurrence Relations
Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The
More informationAlternating Series. 1 n 0 2 n n THEOREM 9.14 Alternating Series Test Let a n > 0. The alternating series. 1 n a n.
0_0905.qxd //0 :7 PM Page SECTION 9.5 Alteratig Series Sectio 9.5 Alteratig Series Use the Alteratig Series Test to determie whether a ifiite series coverges. Use the Alteratig Series Remaider to approximate
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationDisjoint set (Union-Find)
CS124 Lecture 7 Fall 2018 Disjoit set (Uio-Fid) For Kruskal s algorithm for the miimum spaig tree problem, we foud that we eeded a data structure for maitaiig a collectio of disjoit sets. That is, we eed
More informationMCT242: Electronic Instrumentation Lecture 2: Instrumentation Definitions
Faculty of Egieerig MCT242: Electroic Istrumetatio Lecture 2: Istrumetatio Defiitios Overview Measuremet Error Accuracy Precisio ad Mea Resolutio Mea Variace ad Stadard deviatio Fiesse Sesitivity Rage
More informationFREE VIBRATION RESPONSE OF A SYSTEM WITH COULOMB DAMPING
Mechaical Vibratios FREE VIBRATION RESPONSE OF A SYSTEM WITH COULOMB DAMPING A commo dampig mechaism occurrig i machies is caused by slidig frictio or dry frictio ad is called Coulomb dampig. Coulomb dampig
More informationActivity 3: Length Measurements with the Four-Sided Meter Stick
Activity 3: Legth Measuremets with the Four-Sided Meter Stick OBJECTIVE: The purpose of this experimet is to study errors ad the propagatio of errors whe experimetal data derived usig a four-sided meter
More informationPattern Classification, Ch4 (Part 1)
Patter Classificatio All materials i these slides were take from Patter Classificatio (2d ed) by R O Duda, P E Hart ad D G Stork, Joh Wiley & Sos, 2000 with the permissio of the authors ad the publisher
More informationTime-Domain Representations of LTI Systems
2.1 Itroductio Objectives: 1. Impulse resposes of LTI systems 2. Liear costat-coefficiets differetial or differece equatios of LTI systems 3. Bloc diagram represetatios of LTI systems 4. State-variable
More informationSequences, Mathematical Induction, and Recursion. CSE 2353 Discrete Computational Structures Spring 2018
CSE 353 Discrete Computatioal Structures Sprig 08 Sequeces, Mathematical Iductio, ad Recursio (Chapter 5, Epp) Note: some course slides adopted from publisher-provided material Overview May mathematical
More informationDiscrete-Time Systems, LTI Systems, and Discrete-Time Convolution
EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationOBJECTIVES. Chapter 1 INTRODUCTION TO INSTRUMENTATION FUNCTION AND ADVANTAGES INTRODUCTION. At the end of this chapter, students should be able to:
OBJECTIVES Chapter 1 INTRODUCTION TO INSTRUMENTATION At the ed of this chapter, studets should be able to: 1. Explai the static ad dyamic characteristics of a istrumet. 2. Calculate ad aalyze the measuremet
More informationAnalysis of Algorithms. Introduction. Contents
Itroductio The focus of this module is mathematical aspects of algorithms. Our mai focus is aalysis of algorithms, which meas evaluatig efficiecy of algorithms by aalytical ad mathematical methods. We
More informationPattern recognition systems Lab 10 Linear Classifiers and the Perceptron Algorithm
Patter recogitio systems Lab 10 Liear Classifiers ad the Perceptro Algorithm 1. Objectives his lab sessio presets the perceptro learig algorithm for the liear classifier. We will apply gradiet descet ad
More informationGEORGIA INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING
GEORGIA INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING ECE 06 Summer 07 Problem Set #5 Assiged: Jue 3, 07 Due Date: Jue 30, 07 Readig: Chapter 5 o FIR Filters. PROBLEM 5..* (The
More informationMachine Learning Theory (CS 6783)
Machie Learig Theory (CS 6783) Lecture 2 : Learig Frameworks, Examples Settig up learig problems. X : istace space or iput space Examples: Computer Visio: Raw M N image vectorized X = 0, 255 M N, SIFT
More informationMixtures of Gaussians and the EM Algorithm
Mixtures of Gaussias ad the EM Algorithm CSE 6363 Machie Learig Vassilis Athitsos Computer Sciece ad Egieerig Departmet Uiversity of Texas at Arligto 1 Gaussias A popular way to estimate probability desity
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 3
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture 3 Tolstikhi Ilya Abstract I this lecture we will prove the VC-boud, which provides a high-probability excess risk boud for the ERM algorithm whe
More informationsubcaptionfont+=small,labelformat=parens,labelsep=space,skip=6pt,list=0,hypcap=0 subcaption ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, 2/16/2016
subcaptiofot+=small,labelformat=pares,labelsep=space,skip=6pt,list=0,hypcap=0 subcaptio ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, /6/06. Self-cojugate Partitios Recall that, give a partitio λ, we may
More informationChapter 4 : Laplace Transform
4. Itroductio Laplace trasform is a alterative to solve the differetial equatio by the complex frequecy domai ( s = σ + jω), istead of the usual time domai. The DE ca be easily trasformed ito a algebraic
More informationCS / MCS 401 Homework 3 grader solutions
CS / MCS 401 Homework 3 grader solutios assigmet due July 6, 016 writte by Jāis Lazovskis maximum poits: 33 Some questios from CLRS. Questios marked with a asterisk were ot graded. 1 Use the defiitio of
More informationCS322: Network Analysis. Problem Set 2 - Fall 2009
Due October 9 009 i class CS3: Network Aalysis Problem Set - Fall 009 If you have ay questios regardig the problems set, sed a email to the course assistats: simlac@staford.edu ad peleato@staford.edu.
More information3.2 Properties of Division 3.3 Zeros of Polynomials 3.4 Complex and Rational Zeros of Polynomials
Math 60 www.timetodare.com 3. Properties of Divisio 3.3 Zeros of Polyomials 3.4 Complex ad Ratioal Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered
More informationExpectation-Maximization Algorithm.
Expectatio-Maximizatio Algorithm. Petr Pošík Czech Techical Uiversity i Prague Faculty of Electrical Egieerig Dept. of Cyberetics MLE 2 Likelihood.........................................................................................................
More informationELEC1200: A System View of Communications: from Signals to Packets Lecture 3
ELEC2: A System View of Commuicatios: from Sigals to Packets Lecture 3 Commuicatio chaels Discrete time Chael Modelig the chael Liear Time Ivariat Systems Step Respose Respose to sigle bit Respose to geeral
More informationClassification of problem & problem solving strategies. classification of time complexities (linear, logarithmic etc)
Classificatio of problem & problem solvig strategies classificatio of time complexities (liear, arithmic etc) Problem subdivisio Divide ad Coquer strategy. Asymptotic otatios, lower boud ad upper boud:
More informationBoosting. Professor Ameet Talwalkar. Professor Ameet Talwalkar CS260 Machine Learning Algorithms March 1, / 32
Boostig Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machie Learig Algorithms March 1, 2017 1 / 32 Outlie 1 Admiistratio 2 Review of last lecture 3 Boostig Professor Ameet Talwalkar CS260
More informationLecture 11: Pseudorandom functions
COM S 6830 Cryptography Oct 1, 2009 Istructor: Rafael Pass 1 Recap Lecture 11: Pseudoradom fuctios Scribe: Stefao Ermo Defiitio 1 (Ge, Ec, Dec) is a sigle message secure ecryptio scheme if for all uppt
More informationf(x) dx as we do. 2x dx x also diverges. Solution: We compute 2x dx lim
Math 3, Sectio 2. (25 poits) Why we defie f(x) dx as we do. (a) Show that the improper itegral diverges. Hece the improper itegral x 2 + x 2 + b also diverges. Solutio: We compute x 2 + = lim b x 2 + =
More informationThe Riemann Zeta Function
Physics 6A Witer 6 The Riema Zeta Fuctio I this ote, I will sketch some of the mai properties of the Riema zeta fuctio, ζ(x). For x >, we defie ζ(x) =, x >. () x = For x, this sum diverges. However, we
More informationLecture 9: Hierarchy Theorems
IAS/PCMI Summer Sessio 2000 Clay Mathematics Udergraduate Program Basic Course o Computatioal Complexity Lecture 9: Hierarchy Theorems David Mix Barrigto ad Alexis Maciel July 27, 2000 Most of this lecture
More informationSignal Processing in Mechatronics. Lecture 3, Convolution, Fourier Series and Fourier Transform
Sigal Processig i Mechatroics Summer semester, 1 Lecture 3, Covolutio, Fourier Series ad Fourier rasform Dr. Zhu K.P. AIS, UM 1 1. Covolutio Covolutio Descriptio of LI Systems he mai premise is that the
More informationAN ALMOST LINEAR RECURRENCE. Donald E. Knuth Calif. Institute of Technology, Pasadena, Calif.
AN ALMOST LINEAR RECURRENCE Doald E. Kuth Calif. Istitute of Techology, Pasadea, Calif. form A geeral liear recurrece with costat coefficiets has the U 0 = a l* U l = a 2 " ' " U r - l = a r ; u = b, u,
More informationInformation-based Feature Selection
Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with
More informationCS321. Numerical Analysis and Computing
CS Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 September 8 5 What is the Root May physical system ca
More informationSummary: CORRELATION & LINEAR REGRESSION. GC. Students are advised to refer to lecture notes for the GC operations to obtain scatter diagram.
Key Cocepts: 1) Sketchig of scatter diagram The scatter diagram of bivariate (i.e. cotaiig two variables) data ca be easily obtaied usig GC. Studets are advised to refer to lecture otes for the GC operatios
More informationCS537. Numerical Analysis and Computing
CS57 Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 Jauary 9 9 What is the Root May physical system ca be
More informationAppendix: The Laplace Transform
Appedix: The Laplace Trasform The Laplace trasform is a powerful method that ca be used to solve differetial equatio, ad other mathematical problems. Its stregth lies i the fact that it allows the trasformatio
More informationLecture 3. Digital Signal Processing. Chapter 3. z-transforms. Mikael Swartling Nedelko Grbic Bengt Mandersson. rev. 2016
Lecture 3 Digital Sigal Processig Chapter 3 z-trasforms Mikael Swartlig Nedelko Grbic Begt Madersso rev. 06 Departmet of Electrical ad Iformatio Techology Lud Uiversity z-trasforms We defie the z-trasform
More informationEE / EEE SAMPLE STUDY MATERIAL. GATE, IES & PSUs Signal System. Electrical Engineering. Postal Correspondence Course
Sigal-EE Postal Correspodece Course 1 SAMPLE STUDY MATERIAL Electrical Egieerig EE / EEE Postal Correspodece Course GATE, IES & PSUs Sigal System Sigal-EE Postal Correspodece Course CONTENTS 1. SIGNAL
More informationSignals and Systems. Problem Set: From Continuous-Time to Discrete-Time
Sigals ad Systems Problem Set: From Cotiuous-Time to Discrete-Time Updated: October 5, 2017 Problem Set Problem 1 - Liearity ad Time-Ivariace Cosider the followig systems ad determie whether liearity ad
More informationSignal Processing. Lecture 02: Discrete Time Signals and Systems. Ahmet Taha Koru, Ph. D. Yildiz Technical University.
Sigal Processig Lecture 02: Discrete Time Sigals ad Systems Ahmet Taha Koru, Ph. D. Yildiz Techical Uiversity 2017-2018 Fall ATK (YTU) Sigal Processig 2017-2018 Fall 1 / 51 Discrete Time Sigals Discrete
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationRun-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE
Geeral e Image Coder Structure Motio Video (s 1,s 2,t) or (s 1,s 2 ) Natural Image Samplig A form of data compressio; usually lossless, but ca be lossy Redudacy Removal Lossless compressio: predictive
More informationUnit 6: Sequences and Series
AMHS Hoors Algebra 2 - Uit 6 Uit 6: Sequeces ad Series 26 Sequeces Defiitio: A sequece is a ordered list of umbers ad is formally defied as a fuctio whose domai is the set of positive itegers. It is commo
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam will cover.-.9. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for you
More informationMassachusetts Institute of Technology
Massachusetts Istitute of Techology 6.867 Machie Learig, Fall 6 Problem Set : Solutios. (a) (5 poits) From the lecture otes (Eq 4, Lecture 5), the optimal parameter values for liear regressio give the
More informationVector Quantization: a Limiting Case of EM
. Itroductio & defiitios Assume that you are give a data set X = { x j }, j { 2,,, }, of d -dimesioal vectors. The vector quatizatio (VQ) problem requires that we fid a set of prototype vectors Z = { z
More informationRegression with quadratic loss
Regressio with quadratic loss Maxim Ragisky October 13, 2015 Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X,Y, where, as before,
More informationSolution of Linear Constant-Coefficient Difference Equations
ECE 38-9 Solutio of Liear Costat-Coefficiet Differece Equatios Z. Aliyazicioglu Electrical ad Computer Egieerig Departmet Cal Poly Pomoa Solutio of Liear Costat-Coefficiet Differece Equatios Example: Determie
More informationME NUMERICAL METHODS Fall 2007
ME - 310 NUMERICAL METHODS Fall 2007 Group 02 Istructor: Prof. Dr. Eres Söylemez (Rm C205, email:eres@metu.edu.tr ) Class Hours ad Room: Moday 13:40-15:30 Rm: B101 Wedesday 12:40-13:30 Rm: B103 Course
More informationWHAT IS THE PROBABILITY FUNCTION FOR LARGE TSUNAMI WAVES? ABSTRACT
WHAT IS THE PROBABILITY FUNCTION FOR LARGE TSUNAMI WAVES? Harold G. Loomis Hoolulu, HI ABSTRACT Most coastal locatios have few if ay records of tsuami wave heights obtaied over various time periods. Still
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationIntroduction to Artificial Intelligence CAP 4601 Summer 2013 Midterm Exam
Itroductio to Artificial Itelligece CAP 601 Summer 013 Midterm Exam 1. Termiology (7 Poits). Give the followig task eviromets, eter their properties/characteristics. The properties/characteristics of the
More informationTitle: Damage Identification of Structures Based on Pattern Classification Using Limited Number of Sensors
Cover page Title: Damage Idetificatio of Structures Based o Patter Classificatio Usig Limited Number of Sesors Authors: Yuyi QIAN Akira MITA PAPER DEADLINE: **JULY, ** PAPER LENGTH: **8 PAGES MAXIMUM **
More information-ORDER CONVERGENCE FOR FINDING SIMPLE ROOT OF A POLYNOMIAL EQUATION
NEW NEWTON-TYPE METHOD WITH k -ORDER CONVERGENCE FOR FINDING SIMPLE ROOT OF A POLYNOMIAL EQUATION R. Thukral Padé Research Cetre, 39 Deaswood Hill, Leeds West Yorkshire, LS7 JS, ENGLAND ABSTRACT The objective
More informationOlli Simula T / Chapter 1 3. Olli Simula T / Chapter 1 5
Sigals ad Systems Sigals ad Systems Sigals are variables that carry iformatio Systemstake sigals as iputs ad produce sigals as outputs The course deals with the passage of sigals through systems T-6.4
More informationResearch Article Nonautonomous Discrete Neuron Model with Multiple Periodic and Eventually Periodic Solutions
Discrete Dyamics i Nature ad Society Volume 21, Article ID 147282, 6 pages http://dx.doi.org/1.11/21/147282 Research Article Noautoomous Discrete Neuro Model with Multiple Periodic ad Evetually Periodic
More information