CHALMERS, GÖTEBORGS UNIVERSITET. SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
|
|
- Byron Stevenson
- 5 years ago
- Views:
Transcription
1 CHALMERS, GÖTEBORGS UNIVERSITET SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 35, FIM 72 GU, PhD Tme: Place: Teachers: Allowed materal: Not allowed: January 2, 28, at SB Multsal Bernhard Mehlg, (moble Johan Fres, (moble, vsts once at 9 Mathematcs Handbook for Scence and Engneerng Any other wrtten materal, calculator Maxmum score on ths exam: 2 ponts. Maxmum score for homework problems: 2 ponts. To pass the course t s necessary to score at least 5 ponts on ths wrtten exam. CTH 4 passed; 7.5 grade 4; 22 grade 5, GU 4 grade G; 2 grade VG.. Recognton of one pattern. a Defne Q (µ,ν = =42 = ζ (µ ζ (ν. ( = ζ (ν, and wth f ζ (µ The bt constrbutes wth + to Q (µ,ν f ζ (µ. Snce the number of bts are 42, we have Q (µ,ν = 42 2 H (µ,ν, where ζ (ν H (µ,ν s the number of bts that are dfferent n pattern µ and pattern ν (the Hammng dstance. We fnd: H (, = Q (, = 42 H (,2 = Q (,2 = 22 H (,3 = 2 Q (,3 = 38 H (,4 = 42 Q (,4 = 42 H (,5 = 2 Q (,5 =
2 H (2, = H (,2 = Q (2, = 22 H (2,2 = Q (2,2 = 42 H (2,3 = Q (2,3 = 22 H (2,4 = 42 H (2, = 32 Q (2,4 = 22 H (2,5 = Q (2,5 = 2 b We have that b (ν = = 42 ζ( From a, we have that: b ( b (2 b (3 b (4 b (5 w ζ (ν = ζ ( ζ (ν ( ζ ( ζ ( + ζ (2 ζ (2 ζ (ν ζ(2 ζ (2 ζ (ν = 42 ζ( Q (,ν + 42 ζ(2 Q (2,ν. (2 = Q(, 42 ζ( + Q(2, 42 ζ(2 = ζ ( ζ(2, = Q(,2 42 ζ( + Q(2,2 42 ζ(2 = ζ( + ζ (2, = Q(,3 42 ζ( + Q(2,3 42 ζ(2 = ζ( ζ(2, = Q(,4 42 ζ( + Q(2,4 42 ζ(2 = ζ ( ζ(2, = Q(,5 42 ζ( + Q(2,5 42 ζ(2 = 2 42 ζ(2. (3 c We have that From b, we fnd that: ζ ( sgn(b ( = ζ (, ζ (2 sgn(b (2 = ζ (2, ζ (3 sgn(b (3 = ζ (, ζ (4 sgn(b (4 = ζ ( = ζ (4, Thus patterns ζ (, ζ (2 and ζ (4 are stable. ζ (5 sgn(b (5 = ζ (2. (4 2
3 2. Lnearly nseparable problem. a In the fgure below ξ (A and ξ (B are to have output and ξ (C and ξ (D are to have output. There s no straght lne that can separate patterns ξ (A and ξ (B from patterns ξ (C and ξ (D. b The trangle corners are: [ ] 4 ξ ( = ξ (2 = [ ] 4 ξ (3 = [ ]. (5 3 Let v = at ξ ( and ξ (2. Ths mples = w ξ ( + w 2 ξ ( 2 θ = 4w θ = w ξ (2 + w 2 ξ (2 2 θ = 4w w 2 θ θ = 4w and w 2 = 4w θ = 8w. (6 We choose w =, w 2 = 8 and θ = 4. 3
4 Let v 2 = at ξ (2 and ξ (3. Ths mples = w2 ξ (2 + w 22 ξ (2 2 θ 2 = 4w 2 w 22 θ 2 = w 2 ξ (3 + w 22 ξ (3 2 θ 2 = 3w 22 θ 2 w 22 = 4w 2 θ 2 = 4w 2 3w 22 w 22 = w 2 and θ 2 = 3w 22. (7 We choose w 2 = w 22 = and θ 2 = 3. Let v 3 = at ξ (3 and ξ (. Ths mples = w 3 ξ (3 + w 32 ξ (3 2 θ 3 = 3w 32 θ 3 = w 3 ξ ( + w 32 ξ ( 2 θ 3 = 4w 3 θ 3 3w 32 = 4w 3 and θ 3 = 3w 32. (8 We choose w 32 = 4, w 3 = 3 and θ 3 = 2. In summary: 8 4 w = and θ = 3. ( The orgn maps to 4 v = H (w θ = H 3 =. ( 2 We know that the orgn maps to v = [,, ] T and that the hdden neurons change values at the dashed lnes: 4
5 Thus we can conclude that the regons n nput space maps to these regons n the hdden space: We want v = [,, ] T to map to and all other possble values of v to map to. The hdden space can be llustrated as ths: W must be normal to the plane passng through the crosses n the pcture 5
6 above. Also, W ponts to v = [,, ] T from v = [,, ] T. We may choose W = =. ( We know that the pont v = [/2,, ] T les on the decson boundary we are lookng for. So /2 W T T = T = 2. (2 3. Backpropagaton. a Let N m denote the number of weghts w (m. Let n m denote the number of hdden unts v (m,µ for =,..., L, let n denote the number of nput unts and let n L denote the number of output unts. Fnd that the number of weghts are L N m = m= and that the number of thresholds are L n m n m, (3 m= L n m. (4 m= 6
7 b v (m,µ = w (p =g ( =g ( ( g ( = g ( ( θ (m + w (m v (m,µ w (m v (m,µ. (5 Usng that p < m, we fnd: v (m,µ =g ( w (m v (m,µ. (6 w (p c From b, we have: But snce p = m, we fnd: v (m,µ = g ( d We have w (L 2 v (m,µ = g ( w (L 2 δ q δ r v (m,µ = g ( + δw (L 2, where δw (L 2 = η H w (L 2 w (m v (m,µ. (7 δ q v (m,µ r. (8. (9 We derve the energy functon: H w (L 2 From 3b and 3c, we have: v (m,µ ( g = ( g = w (L 2 2 = µ µ ( O (µ ζ (µ ( 2 O (µ ζ (µ O (µ w (L 2 w(m v (m,µ f p < m w (p δ q v r (m,µ f p = m. (2 (2 7
8 Defne v (L,µ = O (µ. We have: O (µ w (L 2 = v(l,µ w (L 2 Insert from eq. (2. Use that L 2 < L. } =g ( b (L,µ w (L v (L,µ w (L 2 Insert from eq. (2. Use that L 2 < L. } ( ( =g b (L,µ w (L g b (L,µ w (L v (L 2,µ k k w (L 2 Insert from eq. (2. Use that L 2 = L 2. } ( ( ( =g b (L,µ w (L g b (L,µ w (L k g k =g ( b (L,µ w (L g ( b (L,µ k k w (L q b (L 2,µ δ qk v (L 3,µ r g ( b (L 2,µ q v (L 3,µ r. (22 The update rule s eq. (9 wth the dervatve of the energy functon gven by eqs. (2 and ( True/False questons. Indcate whether the followng statements are true or false. 3-4 correct answers gve 2 ponts, -2 correct answers gve.5 ponts, 9- correct answers gves pont and, 8 correct answers gve.5 ponts and -7 correct answers gve zero ponts. (2 p. You need access to the state of all neurons n a multlayer perceptron when updatng all weghts through backpropagaton. TRUE (the update of a weght n layer depends on the value of the neuron n the layer before. 2. Consder the Hopfeld network. If a pattern s stable t must be an egenvector of the weght matrx.false (due to the step-functon. 3. If you store two orthogonal patterns n a Hopfeld network, they wll always turn out unstable. FALSE (the crosstalk term s zero. 4. Kohonens algorthm learns convex dstrbutons better than concave ones. TRUE (concave corners can cause problems. 5. The number of N-dmensonal Boolean functons s 2 N. FALSE (t s 2 (2N. 6. The weght matrces n a perceptron are symmetrc. FALSE (they may not even be square matrces. 8
9 7. Usng g(b = b as actvaton functon and puttng all thresholds to zero n a multlayer perceptron, allows you to solve some lnearly nseparable problems. FALSE (you have effectvely one weght matrx that s the product of all your orgnal ones. 8. You need at least four radal bass functons for the XOR-problem to be lnearly separable n the space of the radal bass functons. FALSE (two are enough. 9. Consder p > 2 patterns unformly dstrbuted on a crcle. None of the egenvalues of the covarance matrx of the patterns s zero. TRUE (zero egenvalue ndcates patterns on a lne.. Even f the weght vector n Oa s rule equals ts stable steady state at one teraton, t may change n the followng teratons. TRUE (t s only a statstcally steady state.. If your Kohonen network s supposed to learn the dstrbuton P (ξ, t s mportant to generate the patterns ξ (µ before you start tranng the network. FALSE (tranng your network does not affect whch pattern you draw from your dstrbuton. 2. All one-dmensonal Boolean problems are lnearly separable. TRUE (two dfferent ponts can always be separated by a lne. 3. In Kohonen s algorthm, the neurons have fxed postons n the output space. TRUE (t s the weghts, n the nput space, that are updated. 4. Some elements of the covarance matrx are varances. TRUE (the dagonal elements. 9
10 5. Oa s rule. a Insert ξξ T = C s a matrx, so: = δw = ηζ (ξ ζw ξζ = ζζw. (23 ζ = ξ T w = w T ξ : (24 = δw ξξ T w = w T ξξ T ww ξξ T w = w T ξξ T ww. (25 = δw Cw = w T Cww. (26 We see that δw = mples that w s an egenvector of C wth egenvalue λ = w T Cw: (note that w T w = w w. λ = w T Cw = w T λw = λw T w w T w =. (27 b Are the patterns centered? 5 ξ (µ = = (28 µ= 5 µ= ξ (µ 2 = =. (29 So ξ =, and the patterns are centered. Ths means that the covarance matrx s: C = 5 ξ (µ ξ (µ T = ξξ T. (3 5 We have µ= [ ] 36 3 ξ ( ξ ( T = 3 25 [ ] 4 8 ξ (2 ξ (2 T = 8 6 [ ] 4 4 ξ (3 ξ (3 T = 4 4 [ ] 3 ξ (4 ξ (4 T = 3 9 [ ] 25 2 ξ (5 ξ (5 T =. (3 2 6
11 We compute the elements of C: 5C = = 7 5C 2 =5C 2 = = 65 5C 22 = = 7. (32 We fnd that C = [ ] 4 3. ( Maxmal egenvalue: = 4 λ λ = (4 λ2 3 2 = λ 2 28λ = λ 2 28λ + 27 λ = 4 ± = 4 ± 69 = 4 ± 3 λ max = 27. (34 Egenvector u: [ ] [ ] 4 3 u = u 2 [ u u 2 ] u = u 2. (35 So an egenvector correspongng to the largest egenvalue of C s gven by [ ] u = t (36 for an arbtrary t. Ths s the prncpal component.
12 6. General Boolean problems. There was a typo n Eqn. (8 of the exam. The correct equaton s: v (µ f θ + = w ξ (µ > f θ + w ξ (µ. a The soluton uses w = ξ (. Ths means that the th row of the weght matrx w s a vector w ( = ξ ( : From the fgure above, we see that: w (T ξ ( = + + = 3. w (T ξ (µ = + = for µ =, 4 and 3. w (T ξ (µ = = for µ = 2, 5 and 7. w (T ξ (6 = = 3. Usng that θ = 2, we note that: w (T ξ ( θ s > f = µ < f µ. (37 So we have: v (,µ = f = µ f µ. (38 We can understant that the corner µ of the cube of possble nputs s separated from the other corners by that t assgns to the µ th hdden neuron and to the others. 2
13 From Fgure 4 n the exam, we see that there are exactly 4 of the 8 possble nputs ξ (µ that are to be mapped to O µ =. These are ξ (µ for µ = 2, 4, 5 and 7. These nputs wll assgn, respectvely: v (2 =, v (4 =, v (5 =, and v (7 =. (39 The weghts W are now to detect these and only these patterns, so that O (µ = W T w = for µ 2, 4, 5, 7} for µ, 3, 6, 8}. (4 Ths s acheved by lettng: W =. (4 3
14 b The soluton n 6a mples separatng each corner ξ (µ of the cube of nput patterns by lettng v (,µ f = µ =. (42 f µ Thus the soluton requres 2 3 = 8 hdden neurons. The analogous soluton n 2D s to separate each corner of a square, and t requres 2 N = 2 2 = 4 neurons. The decson boundares of the hdden neurons are shown here: 4
CHALMERS, GÖTEBORGS UNIVERSITET. EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
CHALMERS, GÖTEBORGS UNIVERSITET EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 135, FIM 72 GU, PhD Time: Place: Teachers: Allowed material: Not allowed: October 23, 217, at 8 3 12 3 Lindholmen-salar
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationMultilayer Perceptrons and Backpropagation. Perceptrons. Recap: Perceptrons. Informatics 1 CG: Lecture 6. Mirella Lapata
Multlayer Perceptrons and Informatcs CG: Lecture 6 Mrella Lapata School of Informatcs Unversty of Ednburgh mlap@nf.ed.ac.uk Readng: Kevn Gurney s Introducton to Neural Networks, Chapters 5 6.5 January,
More informationMultilayer neural networks
Lecture Multlayer neural networks Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Mdterm exam Mdterm Monday, March 2, 205 In-class (75 mnutes) closed book materal covered by February 25, 205 Multlayer
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationMulti-layer neural networks
Lecture 0 Mult-layer neural networks Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Lnear regresson w Lnear unts f () Logstc regresson T T = w = p( y =, w) = g( w ) w z f () = p ( y = ) w d w d Gradent
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationMultigradient for Neural Networks for Equalizers 1
Multgradent for Neural Netorks for Equalzers 1 Chulhee ee, Jnook Go and Heeyoung Km Department of Electrcal and Electronc Engneerng Yonse Unversty 134 Shnchon-Dong, Seodaemun-Ku, Seoul 1-749, Korea ABSTRACT
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationMath1110 (Spring 2009) Prelim 3 - Solutions
Math 1110 (Sprng 2009) Solutons to Prelm 3 (04/21/2009) 1 Queston 1. (16 ponts) Short answer. Math1110 (Sprng 2009) Prelm 3 - Solutons x a 1 (a) (4 ponts) Please evaluate lm, where a and b are postve numbers.
More informationAdmin NEURAL NETWORKS. Perceptron learning algorithm. Our Nervous System 10/25/16. Assignment 7. Class 11/22. Schedule for the rest of the semester
0/25/6 Admn Assgnment 7 Class /22 Schedule for the rest of the semester NEURAL NETWORKS Davd Kauchak CS58 Fall 206 Perceptron learnng algorthm Our Nervous System repeat untl convergence (or for some #
More informationCSE 252C: Computer Vision III
CSE 252C: Computer Vson III Lecturer: Serge Belonge Scrbe: Catherne Wah LECTURE 15 Kernel Machnes 15.1. Kernels We wll study two methods based on a specal knd of functon k(x, y) called a kernel: Kernel
More informationInternet Engineering. Jacek Mazurkiewicz, PhD Softcomputing. Part 3: Recurrent Artificial Neural Networks Self-Organising Artificial Neural Networks
Internet Engneerng Jacek Mazurkewcz, PhD Softcomputng Part 3: Recurrent Artfcal Neural Networks Self-Organsng Artfcal Neural Networks Recurrent Artfcal Neural Networks Feedback sgnals between neurons Dynamc
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationDynamic Systems on Graphs
Prepared by F.L. Lews Updated: Saturday, February 06, 200 Dynamc Systems on Graphs Control Graphs and Consensus A network s a set of nodes that collaborates to acheve what each cannot acheve alone. A network,
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More informationNon-linear Canonical Correlation Analysis Using a RBF Network
ESANN' proceedngs - European Smposum on Artfcal Neural Networks Bruges (Belgum), 4-6 Aprl, d-sde publ., ISBN -97--, pp. 57-5 Non-lnear Canoncal Correlaton Analss Usng a RBF Network Sukhbnder Kumar, Elane
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationDUE: WEDS FEB 21ST 2018
HOMEWORK # 1: FINITE DIFFERENCES IN ONE DIMENSION DUE: WEDS FEB 21ST 2018 1. Theory Beam bendng s a classcal engneerng analyss. The tradtonal soluton technque makes smplfyng assumptons such as a constant
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationLecture 10 Support Vector Machines. Oct
Lecture 10 Support Vector Machnes Oct - 20-2008 Lnear Separators Whch of the lnear separators s optmal? Concept of Margn Recall that n Perceptron, we learned that the convergence rate of the Perceptron
More informationHopfield networks and Boltzmann machines. Geoffrey Hinton et al. Presented by Tambet Matiisen
Hopfeld networks and Boltzmann machnes Geoffrey Hnton et al. Presented by Tambet Matsen 18.11.2014 Hopfeld network Bnary unts Symmetrcal connectons http://www.nnwj.de/hopfeld-net.html Energy functon The
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More information1 Derivation of Point-to-Plane Minimization
1 Dervaton of Pont-to-Plane Mnmzaton Consder the Chen-Medon (pont-to-plane) framework for ICP. Assume we have a collecton of ponts (p, q ) wth normals n. We want to determne the optmal rotaton and translaton
More informationThis model contains two bonds per unit cell (one along the x-direction and the other along y). So we can rewrite the Hamiltonian as:
1 Problem set #1 1.1. A one-band model on a square lattce Fg. 1 Consder a square lattce wth only nearest-neghbor hoppngs (as shown n the fgure above): H t, j a a j (1.1) where,j stands for nearest neghbors
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationThe Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor
Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationσ τ τ τ σ τ τ τ σ Review Chapter Four States of Stress Part Three Review Review
Chapter Four States of Stress Part Three When makng your choce n lfe, do not neglect to lve. Samuel Johnson Revew When we use matrx notaton to show the stresses on an element The rows represent the axs
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationp 1 c 2 + p 2 c 2 + p 3 c p m c 2
Where to put a faclty? Gven locatons p 1,..., p m n R n of m houses, want to choose a locaton c n R n for the fre staton. Want c to be as close as possble to all the house. We know how to measure dstance
More informationImportant Instructions to the Examiners:
Summer 0 Examnaton Subject & Code: asc Maths (70) Model Answer Page No: / Important Instructons to the Examners: ) The Answers should be examned by key words and not as word-to-word as gven n the model
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationSupplemental document
Electronc Supplementary Materal (ESI) for Physcal Chemstry Chemcal Physcs. Ths journal s the Owner Socetes 01 Supplemental document Behnam Nkoobakht School of Chemstry, The Unversty of Sydney, Sydney,
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationIntroduction to the Introduction to Artificial Neural Network
Introducton to the Introducton to Artfcal Neural Netork Vuong Le th Hao Tang s sldes Part of the content of the sldes are from the Internet (possbly th modfcatons). The lecturer does not clam any onershp
More informationFor all questions, answer choice E) NOTA" means none of the above answers is correct.
0 MA Natonal Conventon For all questons, answer choce " means none of the above answers s correct.. In calculus, one learns of functon representatons that are nfnte seres called power 3 4 5 seres. For
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 89 Fall 206 Introducton to Machne Learnng Fnal Do not open the exam before you are nstructed to do so The exam s closed book, closed notes except your one-page cheat sheet Usage of electronc devces
More informationBOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS
BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all
More informationQuantum Mechanics I - Session 4
Quantum Mechancs I - Sesson 4 Aprl 3, 05 Contents Operators Change of Bass 4 3 Egenvectors and Egenvalues 5 3. Denton....................................... 5 3. Rotaton n D....................................
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationNeural Networks. Perceptrons and Backpropagation. Silke Bussen-Heyen. 5th of Novemeber Universität Bremen Fachbereich 3. Neural Networks 1 / 17
Neural Networks Perceptrons and Backpropagaton Slke Bussen-Heyen Unverstät Bremen Fachberech 3 5th of Novemeber 2012 Neural Networks 1 / 17 Contents 1 Introducton 2 Unts 3 Network structure 4 Snglelayer
More informationResearch Article Green s Theorem for Sign Data
Internatonal Scholarly Research Network ISRN Appled Mathematcs Volume 2012, Artcle ID 539359, 10 pages do:10.5402/2012/539359 Research Artcle Green s Theorem for Sgn Data Lous M. Houston The Unversty of
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More information15 Lagrange Multipliers
15 The Method of s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve physcs equatons), t s used for several ey dervatons n
More information14 Lagrange Multipliers
Lagrange Multplers 14 Lagrange Multplers The Method of Lagrange Multplers s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve
More information10.34 Fall 2015 Metropolis Monte Carlo Algorithm
10.34 Fall 2015 Metropols Monte Carlo Algorthm The Metropols Monte Carlo method s very useful for calculatng manydmensonal ntegraton. For e.g. n statstcal mechancs n order to calculate the prospertes of
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationPattern Classification
Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION) -- applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More information332600_08_1.qxp 4/17/08 11:29 AM Page 481
336_8_.qxp 4/7/8 :9 AM Page 48 8 Complex Vector Spaces 8. Complex Numbers 8. Conjugates and Dvson of Complex Numbers 8.3 Polar Form and DeMovre s Theorem 8.4 Complex Vector Spaces and Inner Products 8.5
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationSupporting Information
Supportng Informaton The neural network f n Eq. 1 s gven by: f x l = ReLU W atom x l + b atom, 2 where ReLU s the element-wse rectfed lnear unt, 21.e., ReLUx = max0, x, W atom R d d s the weght matrx to
More informationReview: Fit a line to N data points
Revew: Ft a lne to data ponts Correlated parameters: L y = a x + b Orthogonal parameters: J y = a (x ˆ x + b For ntercept b, set a=0 and fnd b by optmal average: ˆ b = y, Var[ b ˆ ] = For slope a, set
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationCSC321 Tutorial 9: Review of Boltzmann machines and simulated annealing
CSC321 Tutoral 9: Revew of Boltzmann machnes and smulated annealng (Sldes based on Lecture 16-18 and selected readngs) Yue L Emal: yuel@cs.toronto.edu Wed 11-12 March 19 Fr 10-11 March 21 Outlne Boltzmann
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More informationCausal Diamonds. M. Aghili, L. Bombelli, B. Pilgrim
Causal Damonds M. Aghl, L. Bombell, B. Plgrm Introducton The correcton to volume of a causal nterval due to curvature of spacetme has been done by Myrhem [] and recently by Gbbons & Solodukhn [] and later
More informationEvaluation of classifiers MLPs
Lecture Evaluaton of classfers MLPs Mlos Hausrecht mlos@cs.ptt.edu 539 Sennott Square Evaluaton For any data set e use to test the model e can buld a confuson matrx: Counts of examples th: class label
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationDecision Boundary Formation of Neural Networks 1
Decson Boundary ormaton of Neural Networks C. LEE, E. JUNG, O. KWON, M. PARK, AND D. HONG Department of Electrcal and Electronc Engneerng, Yonse Unversty 34 Shnchon-Dong, Seodaemum-Ku, Seoul 0-749, Korea
More informationSupport Vector Machines CS434
Support Vector Machnes CS434 Lnear Separators Many lnear separators exst that perfectly classfy all tranng examples Whch of the lnear separators s the best? + + + + + + + + + Intuton of Margn Consder ponts
More informationPlease initial the statement below to show that you have read it
EN0: Structural nalyss Exam I Wednesday, March 2, 2005 Dvson of Engneerng rown Unversty NME: General Instructons No collaboraton of any nd s permtted on ths examnaton. You may consult your own wrtten lecture
More informationThe equation of motion of a dynamical system is given by a set of differential equations. That is (1)
Dynamcal Systems Many engneerng and natural systems are dynamcal systems. For example a pendulum s a dynamcal system. State l The state of the dynamcal system specfes t condtons. For a pendulum n the absence
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More informationEigenvalues of Random Graphs
Spectral Graph Theory Lecture 2 Egenvalues of Random Graphs Danel A. Spelman November 4, 202 2. Introducton In ths lecture, we consder a random graph on n vertces n whch each edge s chosen to be n the
More informationCSCE 790S Background Results
CSCE 790S Background Results Stephen A. Fenner September 8, 011 Abstract These results are background to the course CSCE 790S/CSCE 790B, Quantum Computaton and Informaton (Sprng 007 and Fall 011). Each
More informationCHAPTER-5 INFORMATION MEASURE OF FUZZY MATRIX AND FUZZY BINARY RELATION
CAPTER- INFORMATION MEASURE OF FUZZY MATRI AN FUZZY BINARY RELATION Introducton The basc concept of the fuzz matr theor s ver smple and can be appled to socal and natural stuatons A branch of fuzz matr
More informationFUZZY FINITE ELEMENT METHOD
FUZZY FINITE ELEMENT METHOD RELIABILITY TRUCTURE ANALYI UING PROBABILITY 3.. Maxmum Normal tress Internal force s the shear force, V has a magntude equal to the load P and bendng moment, M. Bendng moments
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More informationMath 217 Fall 2013 Homework 2 Solutions
Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has
More informationModel of Neurons. CS 416 Artificial Intelligence. Early History of Neural Nets. Cybernetics. McCulloch-Pitts Neurons. Hebbian Modification.
Page 1 Model of Neurons CS 416 Artfcal Intellgence Lecture 18 Neural Nets Chapter 20 Multple nputs/dendrtes (~10,000!!!) Cell body/soma performs computaton Sngle output/axon Computaton s typcally modeled
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationThe Expectation-Maximization Algorithm
The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationχ x B E (c) Figure 2.1.1: (a) a material particle in a body, (b) a place in space, (c) a configuration of the body
Secton.. Moton.. The Materal Body and Moton hyscal materals n the real world are modeled usng an abstract mathematcal entty called a body. Ths body conssts of an nfnte number of materal partcles. Shown
More informationMATH 567: Mathematical Techniques in Data Science Lab 8
1/14 MATH 567: Mathematcal Technques n Data Scence Lab 8 Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 11, 2017 Recall We have: a (2) 1 = f(w (1) 11 x 1 + W (1) 12 x 2 + W
More informationProblem Points Score Total 100
Physcs 450 Solutons of Sample Exam I Problem Ponts Score 1 8 15 3 17 4 0 5 0 Total 100 All wor must be shown n order to receve full credt. Wor must be legble and comprehensble wth answers clearly ndcated.
More information