Solving optimal margin classifier
|
|
|
- Daniella Newman
- 5 years ago
- Views:
Transcription
1 Solvg optal arg classfer Recall our opt proble: s s equvalet to Wrte te Lagraga: Recall tat * ca be reforulated as No e solve ts dual proble: b b + s.t a b b + s.t 0 [ ] + b b L * a b b L 0 a b b L 0
2 *** e Dual Proble We ze L t respect to ad b frst: Note tat * ples: Plus *** bac to L ad usg ** e ave: a b b L 0 b 0 L b b 0 L * b L ** e Dual proble cot. No e ave te follog dual opt proble: s s aga a quadratc prograg proble. A global au of ca alas be foud. But at's te bg deal?? Note to tgs:. ca be recovered b. e "erel" a J. s.t. 0 0 K See et More later
3 Support vectors Note te KK codto --- ol a fe 's ca be ozero!! g 0 K 5 0 Class Call te trag data pots ose 's are ozero te support vectors SV Class Support vector aces Oce e ave te Lagrage ultplers { } e ca recostruct te paraeter vector as a egted cobato of te trag eaples: SV For testg t a e data z Copute z + b SV z + b ad classf z as class f te su s postve ad class oterse Note: eed ot be fored eplctl 3
4 Iterpretato of support vector aces e optal s a lear cobato of a sall uber of data pots. s sparse represetato ca be veed as data copresso as te costructo of NN classfer o copute te egts { } ad to use support vector aces e eed to specf ol te er products or erel betee te eaples We ae decsos b coparg eac e eaple z t ol te support vectors: * sg SV z + b No-learl Separable Probles Class Class We allo error ξ classfcato; t s based o te output of te dscrat fucto +b ξ approates te uber of sclassfed saples 4
5 5 Soft Marg Hperplae No e ave a slgtl dfferet opt proble: ξ are slac varables optzato Note tat ξ 0 f tere s o error for ξ s a upper boud of te uber of errors C : tradeoff paraeter betee error ad arg s.t b + 0 ξ ξ + b C ξ e Optzato Proble e dual of ts e costraed optzato proble s s s ver slar to te optzato proble te lear separable case ecept tat tere s a upper boud C o o Oce aga a QP solver ca be used to fd a J. 0 s.t. C 0 K
6 Eteso to No-lear Decso Boudar So far e ave ol cosdered large-arg classfer t a lear decso boudar Ho to geeralze t to becoe olear? Ke dea: trasfor to a ger desoal space to ae lfe easer Iput space: te space te pot are located Feature space: te space of φ after trasforato W trasfor? Lear operato te feature space s equvalet to o-lear operato put space Classfcato ca becoe easer t a proper trasforato. rasforg te Data Iput space φ. Coputato te feature space ca be costl because t s g desoal e feature space s tpcall fte-desoal! e erel trc coes to rescue φ φ φ φ φ φ φ φ φ φ φ φ φ φ φ φ φ φ Feature space Note: feature space s of ger deso ta te put space practce 6
7 7 e Kerel rc Recall te SVM optzato proble e data pots ol appear as er product As log as e ca calculate te er product te feature space e do ot eed te appg eplctl Ma coo geoetrc operatos agles dstaces ca be epressed b er products Defe te erel fucto K b a J. 0 s.t. C 0 K K φ φ A Eaple for feature appg ad erels Cosder a put [ ] Suppose φ. s gve as follos A er product te feature space s So f e defe te erel fucto as follos tere s o eed to carr out φ. eplctl φ ' ' φ φ ' ' K +
8 More eaples of erel fuctos Lear erel e've see t K ' ' Poloal erel e ust sa a eaple ' p K ' + ere p 3 o get te feature vectors e cocateate all pt order poloal ters of te copoets of egted appropratel Radal bass erel K ' ep ' I ts case te feature space cossts of fuctos ad results a oparaetrc classfer. Kerelzed SVM rag: a J K s.t. 0 C 0. K Usg: * sg K SV z + b 8
9 SVM eaples Eaples for No Lear SVMs Gaussa Kerel 9
10 Cross-valdato error e leave-oe-out cross-valdato error does ot deped o te desoalt of te feature space but ol o te # of support vectors! # support vectors Leave - oe - out CV error # of trag eaples Mace Learg 0-70/5 70/ Fall 006 Boostg Erc Xg Lecture 9 October Readg: Cap. 4.3 C.B boo 0
11 Ratoale: Cobato of etods ere s o algort tat s alas te ost accurate We ca select sple ea classfcato or regresso etods ad cobe te to a sgle strog etod Dfferet learers use dfferet Algorts Hperparaeters Represetatos Modaltes rag sets Subprobles e proble: o to cobe te Soe earl algorts Boostg b flterg Scapre 990 Ru ea learer o dfferetl fltered eaple sets Cobe ea poteses Requres oledge o te perforace of ea learer Boostg b aort Freud 995 Ru ea learer o egted eaple set Cobe ea poteses learl Requres oledge o te perforace of ea learer Baggg Brea 996 Ru ea learer o bootstrap replcates of te trag set Average ea poteses Reduces varace
12 Cobato of classfers Suppose e ave a fal of copoet classfers geeratg ± labels suc as decso stups: ere θ {b} b ; θ sg + Eac decso stup pas atteto to ol a sgle copoet of te put vector Cobato of classfers co d We d le to cobe te sple classfers addtvel so tat te fal classfer s te sg of ˆ ; θ + K+ ; θ ere te votes { } epasze copoet classfers tat ae ore relable predctos ta oters Iportat ssues: at s te crtero tat e are optzg? easure of loss e ould le to estate eac e copoet classfer te sae aer odulart
13 3 Measureet of error Loss fucto: Geeralzato error: Obectve: fd t u geeralzato error Ma boostg dea: ze te eprcal error: e.g. I λ [ ] E L λ N N L ˆ λ Epoetal Loss Oe possble easure of eprcal loss s e cobed classfer based o teratos defes a egted loss crtero for te et sple classfer to add eac trag saple s egted b ts "classfablt" or dffcult see b te classfer e ave bult so far { } { } { } { } { } ; ep ; ep ˆ ep ; ˆ ep ˆ ep a W a a θ θ θ ; ; ˆ θ θ + + K
14 4 Learzato of loss fucto We ca splf a bt te estato crtero for te e copoet classfers assug s sall No our eprcal loss crtero reduces to We could coose a e copoet classfer to optze ts egted agreeet { } ; ; ep a a θ θ { } W a W a W ; ; ˆ ep θ θ A possble algort At stage e fd θ* tat aze or at least gve a suffcetl g egted agreeet: eac saple s egted b ts "dffcult" uder te prevousl cobed classfers ore "dffcult" saples receved eaver atteto as te doates te total loss e e go bac ad fd te votes * assocated t te e classfer b zg te orgal egted epoetal loss W ; * θ { } ; ep a W θ
15 Boostg We ave bascall derved a Boostg algort tat sequetall adds e copoet classfers eac traed o reegted trag eaples eac copoet classfer s preseted t a slgtl dfferet proble AdaBoost prelares: e or t oralzed egts W o te trag eaples tall ufor W / te egt reflect te "degree of dffcult" of eac datu o te latest classfer e AdaBoost algort At te t terato e fd a classfer ; θ * for c te egted classfcato error: * ε 0. 5 W ; θ s better ta cage. s s eat to be "eas" --- ea classfer Detere o a votes to assg to te e copoet classfer: 0. 5log ε / ε stroger classfer gets ore votes Update te egts o te trag eaples: W W ep { a ; θ } 5
16 e AdaBoost algort cot d e fal classfer after boostg teratos s gve b te sg of ˆ K + K+ ; θ + + ; θ te votes ere are oralzed for coveece AdaBoost: suar Iput: N eaples S N { N N } a ea base learer θ Italze: equal eaple egts /N for all..n Iterate for t :. tra base learer accordg to egted eaple set t ad obta potess t θ t. copute potess error ε t 3. copute potess egt t 4. update eaple egts for et terato t+ Output: fal potess as a lear cobato of t 6
17 AdaBoost: dataflo dagra AS AS AS Boostg: eaples 7
18 Boostg: eaple cot d Boostg: eaple cot d 8
19 Base Learers Wea learers used practce: Decso stups as parallel splts Decso trees e.g. C4.5 b Qula 996 Mult-laer eural etors Radal bass fucto etors Ca base learers operate o egted eaples? I a cases te ca be odfed to accept egts alog t te eaples I geeral e ca saple te eaples t replaceet accordg to te dstrbuto defed b te egts Boostg perforace e error rate of copoet classfer te decso stups does ot prove uc f at all over te But bot trag ad testg error prove over te! Eve after te trag error of te cobed classfer goes to zero boostg teratos ca stll prove te geeralzato error!! 9
20 W t s org? You ll eed soe learg teor to be covered te et to lectures to uderstad ts full but for o let's ust go over soe g level deas Geeralzato Error: Wt g probablt Geeralzato error s less ta: As goes up our boud becoes orse Boostg sould overft! Eperets est error rag error e Boostg Approac to Mace Learg b Robert E. Scapre 0
21 rag Margs We a vote s tae te ore predctors agreeg te ore cofdet ou are our predcto. Marg for eaple: ; θ + K+ ; θ arg + K+ e arg les [ ] ad s egatve for all sclassfed eaples. Successve boostg teratos prove te aort vote or arg for te trag eaples More Eperets e Boostg Approac to Mace Learg b Robert E. Scapre
22 A Marg Boud For a γ te geeralzato error s less ta: Pr arg + γ O γ d Robert E. Scapre Yoav Freud Peter Bartlett ad Wee Su Lee. Boostg te arg: A e eplaato for te effectveess of votg etods. e Aals of Statstcs 65: It does ot deped o!!! Suar Boostg taes a ea learer ad coverts t to a strog oe Wors b asptotcall zg te eprcal error Effectvel azes te arg of te cobed potess
Support vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
Binary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
CS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
CS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f + + + K d d K k - paraeters eghts
Support vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
Some Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
Generative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
Algorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
An Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
Kernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
Classification : Logistic regression. Generative classification model.
CS 75 Mache Lear Lecture 8 Classfcato : Lostc reresso. Geeratve classfcato model. Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Bar classfcato o classes Y {} Our oal s to lear to classf
Machine Learning. What is a good Decision Boundary? Support Vector Machines
Machne Learnng 0-70/5 70/5-78 78 Sprng 200 Support Vector Machnes Erc Xng Lecture 7 March 5 200 Readng: Chap. 6&7 C.B book and lsted papers Erc Xng @ CMU 2006-200 What s a good Decson Boundar? Consder
CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
CS 2750 Machine Learning Lecture 8. Linear regression. Supervised learning. a set of n examples
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht los@cs.tt.eu 59 Seott Square Suervse learg Data: D { D D.. D} a set of eales D s a ut vector of sze s the esre outut gve b a teacher Obectve: lear
Supervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
D. L. Bricker, 2002 Dept of Mechanical & Industrial Engineering The University of Iowa. CPL/XD 12/10/2003 page 1
D. L. Brcker, 2002 Dept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/XD 2/0/2003 page Capactated Plat Locato Proble: Mze FY + C X subject to = = j= where Y = j= X D, j =, j X SY, =,... X 0, =,
Machine Learning. Support Vector Machines. Eric Xing. Lecture 4, August 12, Reading: Eric CMU,
Machne Learnng Support Vector Machnes Erc Xng Lecture 4 August 2 200 Readng: Erc Xng @ CMU 2006-200 Erc Xng @ CMU 2006-200 2 What s a good Decson Boundar? Wh e a have such boundares? Irregular dstrbuton
A Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming
Aerca Joural of Operatos Research, 4, 4, 33-339 Publshed Ole Noveber 4 ScRes http://wwwscrporg/oural/aor http://ddoorg/436/aor4463 A Pealty Fucto Algorth wth Obectve Paraeters ad Costrat Pealty Paraeter
ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
Chapter 8 Heteroskedasticity
Chapter 8 Heteroskedastct I the ultple regresso odel Xβ + ε, t s assued that e, V ( ε) I, Var( ε ), Cov( εε ), j,,, j I ths case, the dagoal eleets of covarace atrx of ε are sae dcatg that the varace of
KURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS. Peter J. Wilcoxen. Impact Research Centre, University of Melbourne.
KURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS by Peter J. Wlcoxe Ipact Research Cetre, Uversty of Melboure Aprl 1989 Ths paper descrbes a ethod that ca be used to resolve cossteces
Regression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
Recap: the SVM problem
Machne Learnng 0-70/5-78 78 Fall 0 Advanced topcs n Ma-Margn Margn Learnng Erc Xng Lecture 0 Noveber 0 Erc Xng @ CMU 006-00 Recap: the SVM proble We solve the follong constraned opt proble: a s.t. J 0
ECE 194C Target Classification in Sensor Networks Problem. Fundamental problem in pattern recognition.
ECE 94C arget Classfcato Sesor Netorks.ece.ucsb.edu/Facult/Ilts/ece94c Proble Gve sgature of a target, e.g. sesc, acoustc, vdeo. Detere hch categor the sgature belogs to Fudaetal proble patter recogto.
Some results and conjectures about recurrence relations for certain sequences of binomial sums.
Soe results ad coectures about recurrece relatos for certa sequeces of boal sus Joha Cgler Faultät für Matheat Uverstät We A-9 We Nordbergstraße 5 Joha Cgler@uveacat Abstract I a prevous paper [] I have
Capacitated Plant Location Problem:
. L. Brcker, 2002 ept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/ 5/29/2002 page CPL/ 5/29/2002 page 2 Capactated Plat Locato Proble: where Mze F + C subect to = = =, =, S, =,... 0, =, ; =,
Linear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
Machine Learning. Support Vector Machines. Eric Xing , Fall Lecture 9, October 6, 2015
Machne Learnng 0-70 Fall 205 Support Vector Machnes Erc Xng Lecture 9 Octoer 6 205 Readng: Chap. 6&7 C.B ook and lsted papers Erc Xng @ CMU 2006-205 What s a good Decson Boundar? Consder a nar classfcaton
Machine Learning. Introduction to Regression. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Mache Learg CSE6740/CS764/ISYE6740, Fall 0 Itroducto to Regresso Le Sog Lecture 4, August 30, 0 Based o sldes from Erc g, CMU Readg: Chap. 3, CB Mache learg for apartmet hutg Suppose ou are to move to
3D Reconstruction from Image Pairs. Reconstruction from Multiple Views. Computing Scene Point from Two Matching Image Points
D Recostructo fro Iage ars Recostructo fro ultple Ves Dael Deetho Fd terest pots atch terest pots Copute fudaetal atr F Copute caera atrces ad fro F For each atchg age pots ad copute pot scee Coputg Scee
Radial Basis Function Networks
Radal Bass Fucto Netorks Radal Bass Fucto Netorks A specal types of ANN that have three layers Iput layer Hdde layer Output layer Mappg from put to hdde layer s olear Mappg from hdde to output layer s
15-381: Artificial Intelligence. Regression and neural networks (NN)
5-38: Artfcal Itellece Reresso ad eural etorks NN) Mmck the bra I the earl das of AI there as a lot of terest develop models that ca mmc huma thk. Whle o oe ke eactl ho the bra orks ad, eve thouh there
Construction of Composite Indices in Presence of Outliers
Costructo of Coposte dces Presece of Outlers SK Mshra Dept. of Ecoocs North-Easter Hll Uversty Shllog (da). troducto: Oftetes we requre costructg coposte dces by a lear cobato of a uber of dcator varables.
Bayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 17
CS434a/54a: Patter Recogto Prof. Olga Vesler Lecture 7 Today Paraetrc Usupervsed Learg Expectato Maxato (EM) oe of the ost useful statstcal ethods oldest verso 958 (Hartley) seal paper 977 (Depster et
Solutions to problem set ); (, ) (
Solutos to proble set.. L = ( yp p ); L = ( p p ); y y L, L = yp p, p p = yp p, + p [, p ] y y y = yp + p = L y Here we use for eaple that yp, p = yp p p yp = yp, p = yp : factors that coute ca be treated
Outline. Review Numerical Approach. Schedule for April and May. Review Simple Methods. Review Notation and Order
Sstes of Ordnar Dfferental Equatons Aprl, Solvng Sstes of Ordnar Dfferental Equatons Larr Caretto Mecancal Engneerng 9 Nuercal Analss of Engneerng Sstes Aprl, Outlne Revew bascs of nuercal solutons of
Dimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
PGE 310: Formulation and Solution in Geosystems Engineering. Dr. Balhoff. Interpolation
PGE 30: Formulato ad Soluto Geosystems Egeerg Dr. Balhoff Iterpolato Numercal Methods wth MATLAB, Recktewald, Chapter 0 ad Numercal Methods for Egeers, Chapra ad Caale, 5 th Ed., Part Fve, Chapter 8 ad
Linear regression (cont.) Linear methods for classification
CS 75 Mache Lear Lecture 7 Lear reresso cot. Lear methods for classfcato Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Coeffcet shrae he least squares estmates ofte have lo bas but hh
TESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
Basic Concepts in Numerical Analysis November 6, 2017
Basc Cocepts Nuercal Aalyss Noveber 6, 7 Basc Cocepts Nuercal Aalyss Larry Caretto Mecacal Egeerg 5AB Sear Egeerg Aalyss Noveber 6, 7 Outle Revew last class Mdter Exa Noveber 5 covers ateral o deretal
Uniform DFT Filter Banks 1/27
.. Ufor FT Flter Baks /27 Ufor FT Flter Baks We ll look at 5 versos of FT-based flter baks all but the last two have serous ltatos ad are t practcal. But they gve a ce trasto to the last two versos whch
A Conventional Approach for the Solution of the Fifth Order Boundary Value Problems Using Sixth Degree Spline Functions
Appled Matheatcs, 1, 4, 8-88 http://d.do.org/1.4/a.1.448 Publshed Ole Aprl 1 (http://www.scrp.org/joural/a) A Covetoal Approach for the Soluto of the Ffth Order Boudary Value Probles Usg Sth Degree Sple
CSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
General Method for Calculating Chemical Equilibrium Composition
AE 6766/Setzma Sprg 004 Geeral Metod for Calculatg Cemcal Equlbrum Composto For gve tal codtos (e.g., for gve reactats, coose te speces to be cluded te products. As a example, for combusto of ydroge wt
Outline. Remaining Course Schedule. Review Systems of ODEs. Example. Example Continued. Other Algorithms for Ordinary Differential Equations
ter Nuercal DE Algorts Aprl 8 0 ter Algorts or rdar Deretal Equatos Larr aretto Mecacal Egeerg 09 Nuercal Aalss o Egeerg Sstes Aprl 8 0 utle Scedule Revew sstes o DEs Sprg-ass-daper proble wt two asses
DATA DOMAIN DATA DOMAIN
3//6 Coprght otce: Most ages these sldes are Gozalez ad oods Pretce-Hall Note: ages are [spatall] ostatoar sgals. e eed tools to aalze the locall at dfferet resolutos e ca do ths the data doa or sutable
5. Data Compression. Review of Last Lecture. Outline of the Lecture. Course Overview. Basics of Information Theory: Markku Juntti
: Markku Jutt Overvew The deas of lossless data copresso ad source codg are troduced ad copresso lts are derved. Source The ateral s aly based o Sectos 5. 5.5 of the course book []. Teleco. Laboratory
Polyphase Filters. Section 12.4 Porat
Polyphase Flters Secto.4 Porat .4 Polyphase Flters Polyphase s a way of dog saplg-rate coverso that leads to very effcet pleetatos. But ore tha that, t leads to very geeral vewpots that are useful buldg
A New Method for Solving Fuzzy Linear. Programming by Solving Linear Programming
ppled Matheatcal Sceces Vol 008 o 50 7-80 New Method for Solvg Fuzzy Lear Prograg by Solvg Lear Prograg S H Nasser a Departet of Matheatcs Faculty of Basc Sceces Mazadara Uversty Babolsar Ira b The Research
The Optimal Algorithm. 7. Algorithm-Independent Learning. No Free Lunch theorem. Theorem: No Free Lunch. Aleix M. Martinez
The Optmal Algorthm 7. Algorthm-Idepedet Learg Alex M. Martez alex@ece.osu.edu Hadouts Hadoutsfor forece ECE874, 007 007 I ths course we have defed a large umber of PR algorthms. The obvous questo to as
Introduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
2/20/2013. Topics. Power Flow Part 1 Text: Power Transmission. Power Transmission. Power Transmission. Power Transmission
/0/0 Topcs Power Flow Part Text: 0-0. Power Trassso Revsted Power Flow Equatos Power Flow Proble Stateet ECEGR 45 Power Systes Power Trassso Power Trassso Recall that for a short trassso le, the power
Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
Arithmetic Mean and Geometric Mean
Acta Mathematca Ntresa Vol, No, p 43 48 ISSN 453-6083 Arthmetc Mea ad Geometrc Mea Mare Varga a * Peter Mchalča b a Departmet of Mathematcs, Faculty of Natural Sceces, Costate the Phlosopher Uversty Ntra,
Non-degenerate Perturbation Theory
No-degeerate Perturbato Theory Proble : H E ca't solve exactly. But wth H H H' H" L H E Uperturbed egevalue proble. Ca solve exactly. E Therefore, kow ad. H ' H" called perturbatos Copyrght Mchael D. Fayer,
Multivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
ANALYSIS ON THE NATURE OF THE BASIC EQUATIONS IN SYNERGETIC INTER-REPRESENTATION NETWORK
Far East Joural of Appled Mathematcs Volume, Number, 2008, Pages Ths paper s avalable ole at http://www.pphm.com 2008 Pushpa Publshg House ANALYSIS ON THE NATURE OF THE ASI EQUATIONS IN SYNERGETI INTER-REPRESENTATION
The Mathematics of Portfolio Theory
The Matheatcs of Portfolo Theory The rates of retur of stocks, ad are as follows Market odtos state / scearo) earsh Neutral ullsh Probablty 0. 0.5 0.3 % 5% 9% -3% 3% % 5% % -% Notato: R The retur of stock
Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
Asymptotic Formulas Composite Numbers II
Iteratoal Matematcal Forum, Vol. 8, 203, o. 34, 65-662 HIKARI Ltd, www.m-kar.com ttp://d.do.org/0.2988/mf.203.3854 Asymptotc Formulas Composte Numbers II Rafael Jakmczuk Dvsó Matemátca, Uversdad Nacoal
3.1 Introduction to Multinomial Logit and Probit
ES3008 Ecooetrcs Lecture 3 robt ad Logt - Multoal 3. Itroducto to Multoal Logt ad robt 3. Estato of β 3. Itroducto to Multoal Logt ad robt The ultoal Logt odel s used whe there are several optos (ad therefore
Derivation of the Modified Bi-quintic B-spline Base Functions: An Application to Poisson Equation
Aerca Joural of Coputatoal ad Appled Mateatcs 3 3(): 6-3 DOI:.93/j.ajca.33. Dervato of te Modfed B-qutc B-sple Base Fuctos: A Applcato to Posso Equato S. Kutlua N. M. Yagurlu * Departet of Mateatcs İöü
1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
MULTIOBJECTIVE NONLINEAR FRACTIONAL PROGRAMMING PROBLEMS INVOLVING GENERALIZED d - TYPE-I n -SET FUNCTIONS
THE PUBLIHING HOUE PROCEEDING OF THE ROMANIAN ACADEMY, eres A OF THE ROMANIAN ACADEMY Volue 8, Nuber /27,.- MULTIOBJECTIVE NONLINEAR FRACTIONAL PROGRAMMING PROBLEM INVOLVING GENERALIZED d - TYPE-I -ET
Machine Learning. Support Vector Machines. Eric Xing , Fall Lecture 9, October 8, 2015
Machne Learnng 0-70 Fall 205 Support Vector Machnes Erc Xng Lecture 9 Octoer 8 205 Readng: Chap. 6&7 C.B ook and lsted papers Erc Xng @ CMU 2006-205 What s a good Decson Boundar? Consder a nar classfcaton
Nonlinear Piecewise-Defined Difference Equations with Reciprocal Quadratic Terms
Joural of Matematcs ad Statstcs Orgal Researc Paper Nolear Pecewse-Defed Dfferece Equatos wt Recprocal Quadratc Terms Ramada Sabra ad Saleem Safq Al-Asab Departmet of Matematcs, Faculty of Scece, Jaza
6. Nonparametric techniques
6. Noparametrc techques Motvato Problem: how to decde o a sutable model (e.g. whch type of Gaussa) Idea: just use the orgal data (lazy learg) 2 Idea 1: each data pot represets a pece of probablty P(x)
Idea is to sample from a different distribution that picks points in important regions of the sample space. Want ( ) ( ) ( ) E f X = f x g x dx
Importace Samplg Used for a umber of purposes: Varace reducto Allows for dffcult dstrbutos to be sampled from. Sestvty aalyss Reusg samples to reduce computatoal burde. Idea s to sample from a dfferet
LINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
Numerical Analysis Formulae Booklet
Numercal Aalyss Formulae Booklet. Iteratve Scemes for Systems of Lear Algebrac Equatos:.... Taylor Seres... 3. Fte Dfferece Approxmatos... 3 4. Egevalues ad Egevectors of Matrces.... 3 5. Vector ad Matrx
4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
Department of Mechanical Engineering ME 322 Mechanical Engineering Thermodynamics. Ideal Gas Mixtures. Lecture 31
Departet of echacal Egeerg E 322 echacal Egeerg Therodyacs Ideal Gas xtures Lecture 31 xtures Egeerg Applcatos atural gas ethae, ethae, propae, butae, troge, hydroge, carbo doxde, ad others Refrgerats
Multiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
Fast algorithm for efficient simulation of quantum algorithm gates on classical computer
Fast algort for effcet sulato of quatu algort gates o classcal coputer Sergey A PANFILOV Sergey V LYANOV Ludla V LITVINTSEVA Yaaa Motor Europe NV R&D Offce Va Braate 65 603 Crea (CR) Italy Alexader V YAZENIN
Chapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
New Associative Memories to Recall Real-Valued Patterns
Ne Assocatve Meores to Recall Real-Valued Patters Huberto Sossa Rcardo Barró ad Roberto A Vázquez Cetro de Ivestgacó e Coputacó-IPN Av Jua de Dos Bátz esqua co Mguel Othó de Medzábal Meco Ct 778 Meco hsossa@ccp
Approximation of coherent lower probabilities by 2-monotone measures
6th Iteratoal Syposu o Iprecse Probablty: Theores ad Ther Applcatos Durha Uted Kgdo 9 Approato of eret er probabltes by -otoe easures Adrey G roevch Techologcal Isttute of Souther Federal Uversty Tagarog
Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
Estimation of R= P [Y < X] for Two-parameter Burr Type XII Distribution
![Estimation of R= P [Y < X] for Two-parameter Burr Type XII Distribution](/thumbs/72/66400644.jpg "Estimation of R= P [Y < X] for Two-parameter Burr Type XII Distribution")
World Acade of Scece, Egeerg ad Techolog Iteratoal Joural of Matheatcal ad Coputatoal Sceces Vol:4, No:, Estato of R P [Y < X] for Two-paraeter Burr Tpe XII Dstruto H.Paah, S.Asad Iteratoal Scece Ide,
STA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
SMOOTH SUPPORT VECTOR REGRESSION BASED ON MODIFICATION SPLINE INTERPOLATION
Joural of heoretcal ad Appled Iforato echology 5 th October. Vol. 44 No. 5 - JAI & LLS. All rghts reserved. ISSN: 99-8645 www.att.org E-ISSN: 87-395 OOH SUPPOR VECOR REGRESSION BASED ON ODIFICAION SPLINE
Econometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
Line Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
Standard Deviation for PDG Mass Data
4 Dec 06 Stadard Devato for PDG Mass Data M. J. Gerusa Retred, 47 Clfde Road, Worghall, HP8 9JR, UK. gerusa@aol.co, phoe: +(44) 844 339754 Abstract Ths paper aalyses the data for the asses of eleetary
Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
Overview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
Dimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
Relations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels
Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October
Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier
Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa
Rademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
Unique Common Fixed Point of Sequences of Mappings in G-Metric Space M. Akram *, Nosheen
Vol No : Joural of Facult of Egeerg & echolog JFE Pages 9- Uque Coo Fed Pot of Sequeces of Mags -Metrc Sace M. Ara * Noshee * Deartet of Matheatcs C Uverst Lahore Pasta. Eal: ara7@ahoo.co Deartet of Matheatcs
d dt d d dt dt Also recall that by Taylor series, / 2 (enables use of sin instead of cos-see p.27 of A&F) dsin
Learzato of the Swg Equato We wll cover sectos.5.-.6 ad begg of Secto 3.3 these otes. 1. Sgle mache-fte bus case Cosder a sgle mache coected to a fte bus, as show Fg. 1 below. E y1 V=1./_ Fg. 1 The admttace
Stationary states of atoms and molecules
Statoary states of atos ad olecules I followg wees the geeral aspects of the eergy level structure of atos ad olecules that are essetal for the terpretato ad the aalyss of spectral postos the rotatoal
Power Flow S + Buses with either or both Generator Load S G1 S G2 S G3 S D3 S D1 S D4 S D5. S Dk. Injection S G1
ower Flow uses wth ether or both Geerator Load G G G D D 4 5 D4 D5 ecto G Net Comple ower ecto - D D ecto s egatve sg at load bus = _ G D mlarl Curret ecto = G _ D At each bus coservato of comple power
On Pfaff s solution of the Pfaff problem
Zur Pfaff scen Lösung des Pfaff scen Probles Mat. Ann. 7 (880) 53-530. On Pfaff s soluton of te Pfaff proble By A. MAYER n Lepzg Translated by D. H. Delpenc Te way tat Pfaff adopted for te ntegraton of
SUBCLASS OF HARMONIC UNIVALENT FUNCTIONS ASSOCIATED WITH SALAGEAN DERIVATIVE. Sayali S. Joshi
Faculty of Sceces ad Matheatcs, Uversty of Nš, Serba Avalable at: http://wwwpfacyu/float Float 3:3 (009), 303 309 DOI:098/FIL0903303J SUBCLASS OF ARMONIC UNIVALENT FUNCTIONS ASSOCIATED WIT SALAGEAN DERIVATIVE