Machine Learning Support Vector Machines SVM

Size: px
Start display at page:

Download "Machine Learning Support Vector Machines SVM"

Transcription

1 Mchne Lernng Support Vector Mchnes SVM Lesson 6

2 Dt Clssfcton problem rnng set:, D,,, : nput dt smple {,, K}: clss or lbel of nput rget: Construct functon f : X Y f, D Predcton of clss for n unknon nput * f * Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

3 erest eghbor clssfer he smplest clssfcton method Assumpton: dt belongs to the sme ctegor re neghbors Clssfcton rule: Clssf ccordng to the neghbor(s) Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

4 Clssfcton erest eghbor Clssfer Fnd the nerest neghbor (ccordng to dstnce functon) m m n,, : mn dst, * n * Clss of unknon s smlr to ts neghbor * m Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

5 Fnd k> neghbors Etenson to k- Clssf ccordng to the clss mort Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

6 Vorono dgrm Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

7 Lner Clssfers Κ= clsses Ω, Ω rget: Constructon of hperplne f(,) beteen dt of clsses Decson boundres: f f else f f, then, then re the unknon prmeters Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 7 )

8 lner clssfcton nonlner clssfcton Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 8 )

9 rnng Set D, + > f() lner functon: f Defne seprtng hperplne beteen to clsses + < Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 9 )

10 Queston: Whch s the optmum hperplne tht seprtes better to clsses? Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

11 Queston: Whch s the optmum hperplne tht seprtes better to clsses? Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

12 Queston: Whch s the optmum hperplne tht seprtes better to clsses? Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

13 Queston: Whch s the optmum hperplne tht seprtes better to clsses? Infnte number of solutons! Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

14 Soluton: Mrgnl Mmzton [Boser, Guon, Vpnk 9], [Cortes & Vpnk 95] he optml seprtng hperplne s the one tht gves the mmum mrgn dth Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

15 Mrgnl Mmzton Defnton : Mrgn s the mnmum dstnce of trnng smples to the hperplne mn dstnce Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

16 Mrgnl Mmzton Defnton : Mrgn s the mnmum dstnce of trnng smples to the hperplne m dth Defnton : Mrgn s the mmum dth of boundr round the seprtng hperplne thout coverng n smple Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

17 Mrgnl Mmzton Defnton : Mrgn s the mnmum dstnce of trnng smples to the hperplne Mrgn Defnton : Mrgn s the mmum dth of boundr round the seprtng hperplne thout coverng n smple Wh s the optmum soluton? Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 7 )

18 Mrgnl Mmzton Soluton: Fnd the hperplne tht mmzes the mrgn beteen to clsses. sfe zone Mrgn hs ll mnmze the rsk of clssfer s decson. Also, t ll ncrese the generlzton of clssfer (Vpnck, 963) Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 8 )

19 Dstnce of n pont Mrgn: r( ) + = r mn mn mrgn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 9 )

20 Mrgnl Mmzton Problem mn, ˆ, ˆ : m Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

21 Mrgnl Mmzton Problem mn, ˆ, ˆ : m Soluton: Use sclng fctor k: k mn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

22 Mrgnl Mmzton Problem mn, ˆ, ˆ : m Soluton: Use sclng fctor k: k mn hus mrgn becomes: mn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( )

23 herefore: D: Mrgn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

24 he obectve functon We need to optmze hch s the sme s mnmzng subect to the mrgn requrements ˆ, ˆ : m, s.t. ˆ, ˆ : mn, s.t. Qudrtc Optmzton Problem: mnmze qudrtc functon subect to set of lner neqult constrnts Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

25 SVM rnng Methodolog rnng s formulted s n optmzton problem Dul problem reduces computtonl complet Kernel trck s used to reduce computton Determnton of the model prmeters corresponds to conve optmzton problem. Soluton s strghtforrd (locl soluton s the globl optmum) Mkes use of Lgrnge multplers Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

26 Joseph-Lous Lgrnge (736-83) Optmzton problem th lner neqult constrnts mn Lgrnge functon: f s.t. g c g c L, f g Krush-Khun-ucker (KK) condtons: g c g c c Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

27 Mnmzton Problem: Lgrnge functon: Solvng the Optmzton Problem s.t. mn : ˆ, ˆ, L,, KK condtons Lgrnge multplers Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 7 )

28 Dul Optmzton Problem L ˆ L L,, mnmze Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 8 )

29 Prme problem L,, mnmze Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 9 )

30 Prme problem L,, mnmze ˆ Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

31 s.t. Prme problem Dul problem L,, mnmze ˆ D L mmze, Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

32 Importnt Remrks. he Prme problem hs d+ unknon prmeters tht must be tuned. hese re the lner coeffcents {, }, here d s the dt dmenson. he Dul problem hs unknon prmeters hch re the Lgrnge multplers { =,, }, here s the number of trnng smples. hs s vluble nd convenent for mult-dmensonl dt, here d>>, snce the dul serch spce s sgnfcntl loer n comprson th the prme serch spce. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 3 )

33 . he decson rule for choosng the clss of n unknon smple becomes: hch s lner combnton of dot products of th ll trnng smples, here ech one hs unque eght equl to the Lngrnge multpler. ˆ f f Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 33 )

34 3. Accordng to the KK condtons e hve: hus: or nd nd Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 34 )

35 3. Accordng to the KK condtons e hve: hus: or rnng smples of D th zero eght outsde the mrgn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 35 )

36 3. Accordng to the KK condtons e hve: hus: or rnng smples of D hch re found on the mrgn Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 36 )

37 All trnng smples outsde the mrgn hve = nd the do not pl n sgnfcnt role to the decson. rnng smples over the mrgn hold: Mrgn nd the hve >. hese re clled support vectors nd the pl mportnt role to the decson. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 37 )

38 An emple. Clss (+) 5 = 8 =.6 = 7 = = Support vectors th no-zero vlues ho support the mrgn 4 = 6 =.4 =.8 9 = Clss (-) 3 = Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 38 )

39 4. Kernel trck: Use prtculr representton φ() Ide: he orgnl feture spce s trnsformed nto (usull) lrger feture spce hch ncreses the lkelhood of beng lner seprble. Φ: φ() Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 39 )

40 In the ne spce ll dot products become: hch s clled kernel functon nd specfes smlrt he ne decson rule cn be rtten s: K,, K f f f Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

41 Emples of kernel functons Lner Kernel Polnoml Kernel Gussn ή RBF Kernel Cosne Sgmod... K K, p K K K,, e,, e Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

42 Emple : Construct lner feture spce usng φ() Input Spce Orgnl spce (.) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Kernel spce rnsformed spce Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 4 )

43 Emple o o o o ( ) ( o) ( o) ( ) ( o) ( ) ( ) ( o) X F Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 43 )

44 5. Estmte the constnt term Set of support vectors Substtutng e tke: Summng ll: : S ˆ S S S K s S S, sze of S Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 44 )

45 Applctons Bonformtcs et ctegorzton mnng Hndrtten chrcter recognton Computer Vson me seres nlss.. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 45 )

46 Bonformtcs gene epresson dt Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 46 )

47 et ctegorzton mnng Bg of ords (lecon) Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 47 )

48 onlner SVM he non-seprble cse Mppng dt to hgh dmensonl spce, v φ(), ncrese the lkelhood the dt be seprble. Hoever, ths cnnot be gurnteed. Also, seprtng hperplne mght be susceptble to outlers. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 48 )

49 onlner SVM he non-seprble cse eed to mke the lgorthm ork for nonlnerl seprble cses, s ell s to be less senstve to outlers. Introducton of ulr vrbles ξ hch llo errors,.e. smples beng n erroneous sde of mrgn. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 49 )

50 For n smple : f Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

51 For n smple : f If found n the rght sde (no error), then ξ =. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

52 For n smple : f If found n the rght sde (no error), then ξ =. If found nsde the mrgn but n the rght sde ξ < Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 5 )

53 For n smple : f If found n the rght sde (no error), then ξ =. If found nsde the mrgn but n the rght sde ξ < If found ectl n the hperplne here + = then ξ = Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 53 )

54 For n smple : f If found n the rght sde (no error), then ξ =. If found nsde the mrgn but n the rght sde ξ < If found ectl n the hperplne here + = then ξ = If t s rong clssfed then ξ > Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 54 )

55 We llo mrgn be less thn ξ pls to role of error tolernce for ever smple nd sets up the locl mrgn hch llos mrgn to enter the spce of other clss. Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 55 )

56 onlner SVM Obectve functon: s the totl error tolernce of trnng set Problem: s.t. mn,, C Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 56 )

57 Problem: C L,, C,, mn Lgrnge functon s.t. onlner SVM Lgrnge multplers ( ) Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 57 )

58 ΚΚΤ condtons or or C L,, mnmze he dul form of the problem Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 58 )

59 he dul form of the problem C L,, mnmze L ˆ L C L Prtl dervtves Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 59 )

60 D L mmze, C s.t. Dul form of the problem C L,, mnmze he dul form of the problem Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

61 D L mmze, C s.t. If > then re support vectors: If < C then μ > nd ξ =. It holds: he dul form of the problem Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

62 he dul form of the problem mmze L D s.t. C, If = C then μ = nd ξ >. Smple s nsde the mrgn If ξ then s rght clssfed, If ξ > then s rong clssfed Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 6 )

63 he dul form of the problem mmze LD s.t. C, If = C then μ = nd ξ >. Smple s nsde the mrgn If ξ then s rght clssfed, If ξ > then s rong clssfed Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 63 )

64 he SMO lgorthm J. Pltt, Fst rnng of Support Vector Mchnes usng Sequentl Mnml Optmzton, MI Press (998). Sequentl Mnml Optmzton (SMO) Solvng the dul problem mmze L D s.t. C, Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 64 )

65 SMO lgorthmc structure SMO breks ths problem nto seres of smllest possble sub-problems, hch re then solved sequentll. he smllest problem nvolves to such multplers : hs reduced problem cn be solved nltcll: C 3 nd, m : ˆ D L ˆ f ˆ f ˆ ˆ f ) ( C C C ne ˆ ˆ Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 65 )

66 Emples of non-lner svm clssfcton Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 66 )

67 Mult-clss Clssfcton Workng th more thn clsses o generl schemes one vs. ll clssfers Prse Clssfers Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 67 )

68 One vs. All Clssfers One clssfer for ever clss =,,K Smples of emned clss re postve (lbel +), hle rest smples from ll other K- clsses re negtve emples th lbel -. rnng the K dfferent clssfers nd construct functons: f Decson rule: Clssf n unknon smple to the clss th the mmum functon vlue: d c rg m,..., K f Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 68 )

69 Prse Clssfers One clssfer for ever pr of clsses (, k) rnng the K*(Κ-) clssfers nd construct seprtng functons for ever pr: f k, k, k Decson rule: Clssf n unknon smple to the clss th the most votes mong ll clssfers. In cse of equvlence use the functons vlues for tkng the decson. d Mchne Lernng 7 Computer Scence & Engneerng, Unverst of Ionnn ML6 ( 69 )

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 9

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 9 CS434/541: Pttern Recognton Prof. Olg Veksler Lecture 9 Announcements Fnl project proposl due Nov. 1 1-2 prgrph descrpton Lte Penlt: s 1 pont off for ech d lte Assgnment 3 due November 10 Dt for fnl project

More information

SVMs for regression Non-parametric/instance based classification method

SVMs for regression Non-parametric/instance based classification method S 75 Mchne ernng ecture Mos Huskrecht mos@cs.ptt.edu 539 Sennott Squre SVMs for regresson Non-prmetrc/nstnce sed cssfcton method S 75 Mchne ernng Soft-mrgn SVM Aos some fet on crossng the seprtng hperpne

More information

An Introduction to Support Vector Machines

An Introduction to Support Vector Machines An Introducton to Support Vector Mchnes Wht s good Decson Boundry? Consder two-clss, lnerly seprble clssfcton problem Clss How to fnd the lne (or hyperplne n n-dmensons, n>)? Any de? Clss Per Lug Mrtell

More information

Machine Learning. Support Vector Machines. Le Song. CSE6740/CS7641/ISYE6740, Fall Lecture 8, Sept. 13, 2012 Based on slides from Eric Xing, CMU

Machine Learning. Support Vector Machines. Le Song. CSE6740/CS7641/ISYE6740, Fall Lecture 8, Sept. 13, 2012 Based on slides from Eric Xing, CMU Mchne Lernng CSE6740/CS764/ISYE6740 Fll 0 Support Vector Mchnes Le Song Lecture 8 Sept. 3 0 Bsed on sldes fro Erc Xng CMU Redng: Chp. 6&7 C.B ook Outlne Mu rgn clssfcton Constrned optzton Lgrngn dult Kernel

More information

18.7 Artificial Neural Networks

18.7 Artificial Neural Networks 310 18.7 Artfcl Neurl Networks Neuroscence hs hypotheszed tht mentl ctvty conssts prmrly of electrochemcl ctvty n networks of brn cells clled neurons Ths led McCulloch nd Ptts to devse ther mthemtcl model

More information

SVMs for regression Multilayer neural networks

SVMs for regression Multilayer neural networks Lecture SVMs for regresson Muter neur netors Mos Husrecht mos@cs.ptt.edu 539 Sennott Squre Support vector mchne SVM SVM mmze the mrgn round the seprtng hperpne. he decson functon s fu specfed suset of

More information

Support vector machines for regression

Support vector machines for regression S 75 Mchne ernng ecture 5 Support vector mchnes for regresson Mos Huskrecht mos@cs.ptt.edu 539 Sennott Squre S 75 Mchne ernng he decson oundr: ˆ he decson: Support vector mchnes ˆ α SV ˆ sgn αˆ SV!!: Decson

More information

Principle Component Analysis

Principle Component Analysis Prncple Component Anlyss Jng Go SUNY Bufflo Why Dmensonlty Reducton? We hve too mny dmensons o reson bout or obtn nsghts from o vsulze oo much nose n the dt Need to reduce them to smller set of fctors

More information

UNIVERSITY OF IOANNINA DEPARTMENT OF ECONOMICS. M.Sc. in Economics MICROECONOMIC THEORY I. Problem Set II

UNIVERSITY OF IOANNINA DEPARTMENT OF ECONOMICS. M.Sc. in Economics MICROECONOMIC THEORY I. Problem Set II Mcroeconomc Theory I UNIVERSITY OF IOANNINA DEPARTMENT OF ECONOMICS MSc n Economcs MICROECONOMIC THEORY I Techng: A Lptns (Note: The number of ndctes exercse s dffculty level) ()True or flse? If V( y )

More information

Decomposition of Boolean Function Sets for Boolean Neural Networks

Decomposition of Boolean Function Sets for Boolean Neural Networks Decomposton of Boolen Functon Sets for Boolen Neurl Netorks Romn Kohut,, Bernd Stenbch Freberg Unverst of Mnng nd Technolog Insttute of Computer Scence Freberg (Schs), Germn Outlne Introducton Boolen Neuron

More information

Chapter Newton-Raphson Method of Solving a Nonlinear Equation

Chapter Newton-Raphson Method of Solving a Nonlinear Equation Chpter.4 Newton-Rphson Method of Solvng Nonlner Equton After redng ths chpter, you should be ble to:. derve the Newton-Rphson method formul,. develop the lgorthm of the Newton-Rphson method,. use the Newton-Rphson

More information

Linear and Nonlinear Optimization

Linear and Nonlinear Optimization Lner nd Nonlner Optmzton Ynyu Ye Deprtment of Mngement Scence nd Engneerng Stnford Unversty Stnford, CA 9430, U.S.A. http://www.stnford.edu/~yyye http://www.stnford.edu/clss/msnde/ Ynyu Ye, Stnford, MS&E

More information

Chapter Newton-Raphson Method of Solving a Nonlinear Equation

Chapter Newton-Raphson Method of Solving a Nonlinear Equation Chpter 0.04 Newton-Rphson Method o Solvng Nonlner Equton Ater redng ths chpter, you should be ble to:. derve the Newton-Rphson method ormul,. develop the lgorthm o the Newton-Rphson method,. use the Newton-Rphson

More information

CISE 301: Numerical Methods Lecture 5, Topic 4 Least Squares, Curve Fitting

CISE 301: Numerical Methods Lecture 5, Topic 4 Least Squares, Curve Fitting CISE 3: umercl Methods Lecture 5 Topc 4 Lest Squres Curve Fttng Dr. Amr Khouh Term Red Chpter 7 of the tetoo c Khouh CISE3_Topc4_Lest Squre Motvton Gven set of epermentl dt 3 5. 5.9 6.3 The reltonshp etween

More information

Model Fitting and Robust Regression Methods

Model Fitting and Robust Regression Methods Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Model Fttng nd Robust Regresson Methods CMPE 64: Imge Anlss nd Comuter Vson H o Fttng lnes nd ellses to mge dt Dertment o Comuter Engneerng Unverst

More information

Lecture 4: Piecewise Cubic Interpolation

Lecture 4: Piecewise Cubic Interpolation Lecture notes on Vrtonl nd Approxmte Methods n Appled Mthemtcs - A Perce UBC Lecture 4: Pecewse Cubc Interpolton Compled 6 August 7 In ths lecture we consder pecewse cubc nterpolton n whch cubc polynoml

More information

Rank One Update And the Google Matrix by Al Bernstein Signal Science, LLC

Rank One Update And the Google Matrix by Al Bernstein Signal Science, LLC Introducton Rnk One Updte And the Google Mtrx y Al Bernsten Sgnl Scence, LLC www.sgnlscence.net here re two dfferent wys to perform mtrx multplctons. he frst uses dot product formulton nd the second uses

More information

Dennis Bricker, 2001 Dept of Industrial Engineering The University of Iowa. MDP: Taxi page 1

Dennis Bricker, 2001 Dept of Industrial Engineering The University of Iowa. MDP: Taxi page 1 Denns Brcker, 2001 Dept of Industrl Engneerng The Unversty of Iow MDP: Tx pge 1 A tx serves three djcent towns: A, B, nd C. Ech tme the tx dschrges pssenger, the drver must choose from three possble ctons:

More information

International Journal of Pure and Applied Sciences and Technology

International Journal of Pure and Applied Sciences and Technology Int. J. Pure Appl. Sc. Technol., () (), pp. 44-49 Interntonl Journl of Pure nd Appled Scences nd Technolog ISSN 9-67 Avlle onlne t www.jopst.n Reserch Pper Numercl Soluton for Non-Lner Fredholm Integrl

More information

Definition of Tracking

Definition of Tracking Trckng Defnton of Trckng Trckng: Generte some conclusons bout the moton of the scene, objects, or the cmer, gven sequence of mges. Knowng ths moton, predct where thngs re gong to project n the net mge,

More information

Least squares. Václav Hlaváč. Czech Technical University in Prague

Least squares. Václav Hlaváč. Czech Technical University in Prague Lest squres Václv Hlváč Czech echncl Unversty n Prgue hlvc@fel.cvut.cz http://cmp.felk.cvut.cz/~hlvc Courtesy: Fred Pghn nd J.P. Lews, SIGGRAPH 2007 Course; Outlne 2 Lner regresson Geometry of lest-squres

More information

Review of linear algebra. Nuno Vasconcelos UCSD

Review of linear algebra. Nuno Vasconcelos UCSD Revew of lner lgebr Nuno Vsconcelos UCSD Vector spces Defnton: vector spce s set H where ddton nd sclr multplcton re defned nd stsf: ) +( + ) (+ )+ 5) λ H 2) + + H 6) 3) H, + 7) λ(λ ) (λλ ) 4) H, - + 8)

More information

DCDM BUSINESS SCHOOL NUMERICAL METHODS (COS 233-8) Solutions to Assignment 3. x f(x)

DCDM BUSINESS SCHOOL NUMERICAL METHODS (COS 233-8) Solutions to Assignment 3. x f(x) DCDM BUSINESS SCHOOL NUMEICAL METHODS (COS -8) Solutons to Assgnment Queston Consder the followng dt: 5 f() 8 7 5 () Set up dfference tble through fourth dfferences. (b) Wht s the mnmum degree tht n nterpoltng

More information

LOCAL FRACTIONAL LAPLACE SERIES EXPANSION METHOD FOR DIFFUSION EQUATION ARISING IN FRACTAL HEAT TRANSFER

LOCAL FRACTIONAL LAPLACE SERIES EXPANSION METHOD FOR DIFFUSION EQUATION ARISING IN FRACTAL HEAT TRANSFER Yn, S.-P.: Locl Frctonl Lplce Seres Expnson Method for Dffuson THERMAL SCIENCE, Yer 25, Vol. 9, Suppl., pp. S3-S35 S3 LOCAL FRACTIONAL LAPLACE SERIES EXPANSION METHOD FOR DIFFUSION EQUATION ARISING IN

More information

Which Separator? Spring 1

Which Separator? Spring 1 Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal

More information

Applied Statistics Qualifier Examination

Applied Statistics Qualifier Examination Appled Sttstcs Qulfer Exmnton Qul_june_8 Fll 8 Instructons: () The exmnton contns 4 Questons. You re to nswer 3 out of 4 of them. () You my use ny books nd clss notes tht you mght fnd helpful n solvng

More information

Remember: Project Proposals are due April 11.

Remember: Project Proposals are due April 11. Bonformtcs ecture Notes Announcements Remember: Project Proposls re due Aprl. Clss 22 Aprl 4, 2002 A. Hdden Mrov Models. Defntons Emple - Consder the emple we tled bout n clss lst tme wth the cons. However,

More information

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede. with respect to λ. 1. χ λ χ λ ( ) λ, and thus:

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede. with respect to λ. 1. χ λ χ λ ( ) λ, and thus: More on χ nd errors : uppose tht we re fttng for sngle -prmeter, mnmzng: If we epnd The vlue χ ( ( ( ; ( wth respect to. χ n Tlor seres n the vcnt of ts mnmum vlue χ ( mn χ χ χ χ + + + mn mnmzes χ, nd

More information

Pattern Classification

Pattern Classification Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher

More information

Katholieke Universiteit Leuven Department of Computer Science

Katholieke Universiteit Leuven Department of Computer Science Updte Rules for Weghted Non-negtve FH*G Fctorzton Peter Peers Phlp Dutré Report CW 440, Aprl 006 Ktholeke Unverstet Leuven Deprtment of Computer Scence Celestjnenln 00A B-3001 Heverlee (Belgum) Updte Rules

More information

4. Eccentric axial loading, cross-section core

4. Eccentric axial loading, cross-section core . Eccentrc xl lodng, cross-secton core Introducton We re strtng to consder more generl cse when the xl force nd bxl bendng ct smultneousl n the cross-secton of the br. B vrtue of Snt-Vennt s prncple we

More information

Abhilasha Classes Class- XII Date: SOLUTION (Chap - 9,10,12) MM 50 Mob no

Abhilasha Classes Class- XII Date: SOLUTION (Chap - 9,10,12) MM 50 Mob no hlsh Clsses Clss- XII Dte: 0- - SOLUTION Chp - 9,0, MM 50 Mo no-996 If nd re poston vets of nd B respetvel, fnd the poston vet of pont C n B produed suh tht C B vet r C B = where = hs length nd dreton

More information

SVMs: Duality and Kernel Trick. SVMs as quadratic programs

SVMs: Duality and Kernel Trick. SVMs as quadratic programs 11/17/9 SVMs: Dualt and Kernel rck Machne Learnng - 161 Geoff Gordon MroslavDudík [[[partl ased on sldes of Zv-Bar Joseph] http://.cs.cmu.edu/~ggordon/161/ Novemer 18 9 SVMs as quadratc programs o optmzaton

More information

Support Vector Machines CS434

Support Vector Machines CS434 Support Vector Machnes CS434 Lnear Separators Many lnear separators exst that perfectly classfy all tranng examples Whch of the lnear separators s the best? Intuton of Margn Consder ponts A, B, and C We

More information

In this Chapter. Chap. 3 Markov chains and hidden Markov models. Probabilistic Models. Example: CpG Islands

In this Chapter. Chap. 3 Markov chains and hidden Markov models. Probabilistic Models. Example: CpG Islands In ths Chpter Chp. 3 Mrov chns nd hdden Mrov models Bontellgence bortory School of Computer Sc. & Eng. Seoul Ntonl Unversty Seoul 5-74, Kore The probblstc model for sequence nlyss HMM (hdden Mrov model)

More information

Lecture 36. Finite Element Methods

Lecture 36. Finite Element Methods CE 60: Numercl Methods Lecture 36 Fnte Element Methods Course Coordntor: Dr. Suresh A. Krth, Assocte Professor, Deprtment of Cvl Engneerng, IIT Guwht. In the lst clss, we dscussed on the ppromte methods

More information

Support Vector Machines. Vibhav Gogate The University of Texas at dallas

Support Vector Machines. Vibhav Gogate The University of Texas at dallas Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest

More information

Linear Regression & Least Squares!

Linear Regression & Least Squares! Lner Regresson & Lest Squres Al Borj UWM CS 790 Slde credt: Aykut Erdem Ths&week Lner&regresson&prolem&& ' con0nuous&outputs& ' smple&model Introduce&key&concepts:&& ' loss&func0ons& ' generlz0on& ' op0mz0on&

More information

Partially Observable Systems. 1 Partially Observable Markov Decision Process (POMDP) Formalism

Partially Observable Systems. 1 Partially Observable Markov Decision Process (POMDP) Formalism CS294-40 Lernng for Rootcs nd Control Lecture 10-9/30/2008 Lecturer: Peter Aeel Prtlly Oservle Systems Scre: Dvd Nchum Lecture outlne POMDP formlsm Pont-sed vlue terton Glol methods: polytree, enumerton,

More information

The Number of Rows which Equal Certain Row

The Number of Rows which Equal Certain Row Interntonl Journl of Algebr, Vol 5, 011, no 30, 1481-1488 he Number of Rows whch Equl Certn Row Ahmd Hbl Deprtment of mthemtcs Fcult of Scences Dmscus unverst Dmscus, Sr hblhmd1@gmlcom Abstrct Let be X

More information

Modeling Labor Supply through Duality and the Slutsky Equation

Modeling Labor Supply through Duality and the Slutsky Equation Interntonl Journl of Economc Scences nd Appled Reserch 3 : 111-1 Modelng Lor Supply through Dulty nd the Slutsky Equton Ivn Ivnov 1 nd Jul Dorev Astrct In the present pper n nlyss of the neo-clsscl optmzton

More information

Quiz: Experimental Physics Lab-I

Quiz: Experimental Physics Lab-I Mxmum Mrks: 18 Totl tme llowed: 35 mn Quz: Expermentl Physcs Lb-I Nme: Roll no: Attempt ll questons. 1. In n experment, bll of mss 100 g s dropped from heght of 65 cm nto the snd contner, the mpct s clled

More information

Two Coefficients of the Dyson Product

Two Coefficients of the Dyson Product Two Coeffcents of the Dyson Product rxv:07.460v mth.co 7 Nov 007 Lun Lv, Guoce Xn, nd Yue Zhou 3,,3 Center for Combntorcs, LPMC TJKLC Nnk Unversty, Tnjn 30007, P.R. Chn lvlun@cfc.nnk.edu.cn gn@nnk.edu.cn

More information

SVMs: Duality and Kernel Trick. SVMs as quadratic programs

SVMs: Duality and Kernel Trick. SVMs as quadratic programs /8/9 SVMs: Dualt and Kernel rck Machne Learnng - 6 Geoff Gordon MroslavDudík [[[partl ased on sldes of Zv-Bar Joseph] http://.cs.cmu.edu/~ggordon/6/ Novemer 8 9 SVMs as quadratc programs o optmzaton prolems:

More information

Math 497C Sep 17, Curves and Surfaces Fall 2004, PSU

Math 497C Sep 17, Curves and Surfaces Fall 2004, PSU Mth 497C Sep 17, 004 1 Curves nd Surfces Fll 004, PSU Lecture Notes 3 1.8 The generl defnton of curvture; Fox-Mlnor s Theorem Let α: [, b] R n be curve nd P = {t 0,...,t n } be prtton of [, b], then the

More information

Variable time amplitude amplification and quantum algorithms for linear algebra. Andris Ambainis University of Latvia

Variable time amplitude amplification and quantum algorithms for linear algebra. Andris Ambainis University of Latvia Vrble tme mpltude mplfcton nd quntum lgorthms for lner lgebr Andrs Ambns Unversty of Ltv Tlk outlne. ew verson of mpltude mplfcton;. Quntum lgorthm for testng f A s sngulr; 3. Quntum lgorthm for solvng

More information

CS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015

CS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015 CS 3710: Vsual Recognton Classfcaton and Detecton Adrana Kovashka Department of Computer Scence January 13, 2015 Plan for Today Vsual recognton bascs part 2: Classfcaton and detecton Adrana s research

More information

GAUSS ELIMINATION. Consider the following system of algebraic linear equations

GAUSS ELIMINATION. Consider the following system of algebraic linear equations Numercl Anlyss for Engneers Germn Jordnn Unversty GAUSS ELIMINATION Consder the followng system of lgebrc lner equtons To solve the bove system usng clsscl methods, equton () s subtrcted from equton ()

More information

Logistic Regression Maximum Likelihood Estimation

Logistic Regression Maximum Likelihood Estimation Harvard-MIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla Ohno-Machado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall

More information

CENTROID (AĞIRLIK MERKEZİ )

CENTROID (AĞIRLIK MERKEZİ ) CENTOD (ĞLK MEKEZİ ) centrod s geometrcl concept rsng from prllel forces. Tus, onl prllel forces possess centrod. Centrod s tougt of s te pont were te wole wegt of pscl od or sstem of prtcles s lumped.

More information

A Family of Multivariate Abel Series Distributions. of Order k

A Family of Multivariate Abel Series Distributions. of Order k Appled Mthemtcl Scences, Vol. 2, 2008, no. 45, 2239-2246 A Fmly of Multvrte Abel Seres Dstrbutons of Order k Rupk Gupt & Kshore K. Ds 2 Fculty of Scence & Technology, The Icf Unversty, Agrtl, Trpur, Ind

More information

ORDINARY DIFFERENTIAL EQUATIONS

ORDINARY DIFFERENTIAL EQUATIONS 6 ORDINARY DIFFERENTIAL EQUATIONS Introducton Runge-Kutt Metods Mult-step Metods Sstem o Equtons Boundr Vlue Problems Crcterstc Vlue Problems Cpter 6 Ordnr Derentl Equtons / 6. Introducton In mn engneerng

More information

Chapter 12. Ordinary Differential Equation Boundary Value (BV) Problems

Chapter 12. Ordinary Differential Equation Boundary Value (BV) Problems Chapter. Ordnar Dfferental Equaton Boundar Value (BV) Problems In ths chapter we wll learn how to solve ODE boundar value problem. BV ODE s usuall gven wth x beng the ndependent space varable. p( x) q(

More information

6 Roots of Equations: Open Methods

6 Roots of Equations: Open Methods HK Km Slghtly modfed 3//9, /8/6 Frstly wrtten t Mrch 5 6 Roots of Equtons: Open Methods Smple Fed-Pont Iterton Newton-Rphson Secnt Methods MATLAB Functon: fzero Polynomls Cse Study: Ppe Frcton Brcketng

More information

Utility maximization. Conditions for utility maximization. Consumer theory: Utility maximization and expenditure minimization

Utility maximization. Conditions for utility maximization. Consumer theory: Utility maximization and expenditure minimization Consmer theory: Utlty mmzton nd ependtre mnmzton Lectres n Mcroeconomc Theory Fll 006 Prt 7 0006 GB Ashem ECON430-35 #7 Utlty mmzton Assme prce-tng ehvor n good mrets m p Bdget set : { X p m} where m s

More information

The Schur-Cohn Algorithm

The Schur-Cohn Algorithm Modelng, Estmton nd Otml Flterng n Sgnl Processng Mohmed Njm Coyrght 8, ISTE Ltd. Aendx F The Schur-Cohn Algorthm In ths endx, our m s to resent the Schur-Cohn lgorthm [] whch s often used s crteron for

More information

Maximum Margin Bayesian Networks

Maximum Margin Bayesian Networks Mxmum Mrgn Besn Networks Yuhong Guo Deprtment of Computng Scence Unverst of Albert uhong@cs.ulbert.c Lnl Xu School of Computer Scence Unverst of Wterloo l5xu@cs.uwterloo.c Dle Schuurmns Deprtment of Computng

More information

Intro to Visual Recognition

Intro to Visual Recognition CS 2770: Computer Vson Intro to Vsual Recognton Prof. Adrana Kovashka Unversty of Pttsburgh February 13, 2018 Plan for today What s recognton? a.k.a. classfcaton, categorzaton Support vector machnes Separable

More information

Trade-offs in Optimization of GMDH-Type Neural Networks for Modelling of A Complex Process

Trade-offs in Optimization of GMDH-Type Neural Networks for Modelling of A Complex Process Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) Trde-offs n Optmzton of GDH-Type Neurl Networs for odellng of A Complex Process N.

More information

6.6 The Marquardt Algorithm

6.6 The Marquardt Algorithm 6.6 The Mqudt Algothm lmttons of the gdent nd Tylo expnson methods ecstng the Tylo expnson n tems of ch-sque devtves ecstng the gdent sech nto n tetve mtx fomlsm Mqudt's lgothm utomtclly combnes the gdent

More information

10-701/ Machine Learning, Fall 2005 Homework 3

10-701/ Machine Learning, Fall 2005 Homework 3 10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40

More information

Linear Classification, SVMs and Nearest Neighbors

Linear Classification, SVMs and Nearest Neighbors 1 CSE 473 Lecture 25 (Chapter 18) Lnear Classfcaton, SVMs and Nearest Neghbors CSE AI faculty + Chrs Bshop, Dan Klen, Stuart Russell, Andrew Moore Motvaton: Face Detecton How do we buld a classfer to dstngush

More information

ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE

ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE School of Computer and Communcaton Scences Handout 0 Prncples of Dgtal Communcatons Solutons to Problem Set 4 Mar. 6, 08 Soluton. If H = 0, we have Y = Z Z = Y

More information

Trigonometry. Trigonometry. Solutions. Curriculum Ready ACMMG: 223, 224, 245.

Trigonometry. Trigonometry. Solutions. Curriculum Ready ACMMG: 223, 224, 245. Trgonometry Trgonometry Solutons Currulum Redy CMMG:, 4, 4 www.mthlets.om Trgonometry Solutons Bss Pge questons. Identfy f the followng trngles re rght ngled or not. Trngles,, d, e re rght ngled ndted

More information

Image classification. Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing i them?

Image classification. Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing i them? Image classfcaton Gven te bag-of-features representatons of mages from dfferent classes ow do we learn a model for dstngusng tem? Classfers Learn a decson rule assgnng bag-offeatures representatons of

More information

Classification as a Regression Problem

Classification as a Regression Problem Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class

More information

COMPLEX NUMBER & QUADRATIC EQUATION

COMPLEX NUMBER & QUADRATIC EQUATION MCQ COMPLEX NUMBER & QUADRATIC EQUATION Syllus : Comple numers s ordered prs of rels, Representton of comple numers n the form + nd ther representton n plne, Argnd dgrm, lger of comple numers, modulus

More information

ME 501A Seminar in Engineering Analysis Page 1

ME 501A Seminar in Engineering Analysis Page 1 More oundr-vlue Prolems nd genvlue Prolems n Os ovemer 9, 7 More oundr-vlue Prolems nd genvlue Prolems n Os Lrr retto Menl ngneerng 5 Semnr n ngneerng nlss ovemer 9, 7 Outlne Revew oundr-vlue prolems Soot

More information

Lecture 10 Support Vector Machines II

Lecture 10 Support Vector Machines II Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed

More information

MACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression

MACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression 11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING

More information

Singular Value Decomposition: Theory and Applications

Singular Value Decomposition: Theory and Applications Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real

More information

Machine Learning. What is a good Decision Boundary? Support Vector Machines

Machine Learning. What is a good Decision Boundary? Support Vector Machines Machne Learnng 0-70/5 70/5-78 78 Sprng 200 Support Vector Machnes Erc Xng Lecture 7 March 5 200 Readng: Chap. 6&7 C.B book and lsted papers Erc Xng @ CMU 2006-200 What s a good Decson Boundar? Consder

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

CSE 252C: Computer Vision III

CSE 252C: Computer Vision III CSE 252C: Computer Vson III Lecturer: Serge Belonge Scrbe: Catherne Wah LECTURE 15 Kernel Machnes 15.1. Kernels We wll study two methods based on a specal knd of functon k(x, y) called a kernel: Kernel

More information

Physics 121 Sample Common Exam 2 Rev2 NOTE: ANSWERS ARE ON PAGE 7. Instructions:

Physics 121 Sample Common Exam 2 Rev2 NOTE: ANSWERS ARE ON PAGE 7. Instructions: Physcs 121 Smple Common Exm 2 Rev2 NOTE: ANSWERS ARE ON PAGE 7 Nme (Prnt): 4 Dgt ID: Secton: Instructons: Answer ll 27 multple choce questons. You my need to do some clculton. Answer ech queston on the

More information

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute

More information

Advanced Machine Learning. An Ising model on 2-D image

Advanced Machine Learning. An Ising model on 2-D image Advnced Mchne Lernng Vrtonl Inference Erc ng Lecture 12, August 12, 2009 Redng: Erc ng Erc ng @ CMU, 2006-2009 1 An Isng model on 2-D mge odes encode hdden nformton ptchdentty. They receve locl nformton

More information

Support Vector Machines

Support Vector Machines CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at

More information

Kernel Methods and SVMs Extension

Kernel Methods and SVMs Extension Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general

More information

Mixed Type Duality for Multiobjective Variational Problems

Mixed Type Duality for Multiobjective Variational Problems Ž. ournl of Mthemtcl Anlyss nd Applctons 252, 571 586 2000 do:10.1006 m.2000.7000, vlle onlne t http: www.delrry.com on Mxed Type Dulty for Multoectve Vrtonl Prolems R. N. Mukheree nd Ch. Purnchndr Ro

More information

Non-Linear Data for Neural Networks Training and Testing

Non-Linear Data for Neural Networks Training and Testing Proceedngs of the 4th WSEAS Int Conf on Informton Securty, Communctons nd Computers, Tenerfe, Spn, December 6-8, 005 (pp466-47) Non-Lner Dt for Neurl Networks Trnng nd Testng ABDEL LATIF ABU-DALHOUM MOHAMMED

More information

Dynamic Power Management in a Mobile Multimedia System with Guaranteed Quality-of-Service

Dynamic Power Management in a Mobile Multimedia System with Guaranteed Quality-of-Service Dynmc Power Mngement n Moble Multmed System wth Gurnteed Qulty-of-Servce Qnru Qu, Qng Wu, nd Mssoud Pedrm Dept. of Electrcl Engneerng-Systems Unversty of Southern Clforn Los Angeles CA 90089 Outlne! Introducton

More information

ESCI 342 Atmospheric Dynamics I Lesson 1 Vectors and Vector Calculus

ESCI 342 Atmospheric Dynamics I Lesson 1 Vectors and Vector Calculus ESI 34 tmospherc Dnmcs I Lesson 1 Vectors nd Vector lculus Reference: Schum s Outlne Seres: Mthemtcl Hndbook of Formuls nd Tbles Suggested Redng: Mrtn Secton 1 OORDINTE SYSTEMS n orthonorml coordnte sstem

More information

VECTORS VECTORS VECTORS VECTORS. 2. Vector Representation. 1. Definition. 3. Types of Vectors. 5. Vector Operations I. 4. Equal and Opposite Vectors

VECTORS VECTORS VECTORS VECTORS. 2. Vector Representation. 1. Definition. 3. Types of Vectors. 5. Vector Operations I. 4. Equal and Opposite Vectors 1. Defnton A vetor s n entt tht m represent phsl quntt tht hs mgntude nd dreton s opposed to slr tht ls dreton.. Vetor Representton A vetor n e represented grphll n rrow. The length of the rrow s the mgntude

More information

COS 521: Advanced Algorithms Game Theory and Linear Programming

COS 521: Advanced Algorithms Game Theory and Linear Programming COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton

More information

Work and Energy (Work Done by a Varying Force)

Work and Energy (Work Done by a Varying Force) Lecture 1 Chpter 7 Physcs I 3.5.14 ork nd Energy (ork Done y Vryng Force) Course weste: http://fculty.uml.edu/andry_dnylov/techng/physcsi Lecture Cpture: http://echo36.uml.edu/dnylov13/physcs1fll.html

More information

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture # 15 Scribe: Jieming Mao April 1, 2013

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture # 15 Scribe: Jieming Mao April 1, 2013 COS 511: heoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 15 Scrbe: Jemng Mao Aprl 1, 013 1 Bref revew 1.1 Learnng wth expert advce Last tme, we started to talk about learnng wth expert advce.

More information

INTRODUCTION TO COMPLEX NUMBERS

INTRODUCTION TO COMPLEX NUMBERS INTRODUCTION TO COMPLEX NUMBERS The numers -4, -3, -, -1, 0, 1,, 3, 4 represent the negtve nd postve rel numers termed ntegers. As one frst lerns n mddle school they cn e thought of s unt dstnce spced

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

Lecture 12: Classification

Lecture 12: Classification Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna

More information

Limited Dependent Variables

Limited Dependent Variables Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages

More information

1.2. Linear Variable Coefficient Equations. y + b "! = a y + b " Remark: The case b = 0 and a non-constant can be solved with the same idea as above.

1.2. Linear Variable Coefficient Equations. y + b ! = a y + b  Remark: The case b = 0 and a non-constant can be solved with the same idea as above. 1 12 Liner Vrible Coefficient Equtions Section Objective(s): Review: Constnt Coefficient Equtions Solving Vrible Coefficient Equtions The Integrting Fctor Method The Bernoulli Eqution 121 Review: Constnt

More information

Effects of polarization on the reflected wave

Effects of polarization on the reflected wave Lecture Notes. L Ros PPLIED OPTICS Effects of polrzton on the reflected wve Ref: The Feynmn Lectures on Physcs, Vol-I, Secton 33-6 Plne of ncdence Z Plne of nterfce Fg. 1 Y Y r 1 Glss r 1 Glss Fg. Reflecton

More information

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

6. Chemical Potential and the Grand Partition Function

6. Chemical Potential and the Grand Partition Function 6. Chemcl Potentl nd the Grnd Prtton Functon ome Mth Fcts (see ppendx E for detls) If F() s n nlytc functon of stte vrles nd such tht df d pd then t follows: F F p lso snce F p F we cn conclude: p In other

More information

p 1 c 2 + p 2 c 2 + p 3 c p m c 2

p 1 c 2 + p 2 c 2 + p 3 c p m c 2 Where to put a faclty? Gven locatons p 1,..., p m n R n of m houses, want to choose a locaton c n R n for the fre staton. Want c to be as close as possble to all the house. We know how to measure dstance

More information

Multivariate Ratio Estimation With Known Population Proportion Of Two Auxiliary Characters For Finite Population

Multivariate Ratio Estimation With Known Population Proportion Of Two Auxiliary Characters For Finite Population Multvarate Rato Estmaton Wth Knon Populaton Proporton Of To Auxlar haracters For Fnte Populaton *Raesh Sngh, *Sachn Mal, **A. A. Adeara, ***Florentn Smarandache *Department of Statstcs, Banaras Hndu Unverst,Varanas-5,

More information

LAPLACE TRANSFORM SOLUTION OF THE PROBLEM OF TIME-FRACTIONAL HEAT CONDUCTION IN A TWO-LAYERED SLAB

LAPLACE TRANSFORM SOLUTION OF THE PROBLEM OF TIME-FRACTIONAL HEAT CONDUCTION IN A TWO-LAYERED SLAB Journl of Appled Mthemtcs nd Computtonl Mechncs 5, 4(4), 5-3 www.mcm.pcz.pl p-issn 99-9965 DOI:.75/jmcm.5.4. e-issn 353-588 LAPLACE TRANSFORM SOLUTION OF THE PROBLEM OF TIME-FRACTIONAL HEAT CONDUCTION

More information

Demand. Demand and Comparative Statics. Graphically. Marshallian Demand. ECON 370: Microeconomic Theory Summer 2004 Rice University Stanley Gilbert

Demand. Demand and Comparative Statics. Graphically. Marshallian Demand. ECON 370: Microeconomic Theory Summer 2004 Rice University Stanley Gilbert Demnd Demnd nd Comrtve Sttcs ECON 370: Mcroeconomc Theory Summer 004 Rce Unversty Stnley Glbert Usng the tools we hve develoed u to ths ont, we cn now determne demnd for n ndvdul consumer We seek demnd

More information

Linear discriminants. Nuno Vasconcelos ECE Department, UCSD

Linear discriminants. Nuno Vasconcelos ECE Department, UCSD Lnear dscrmnants Nuno Vasconcelos ECE Department UCSD Classfcaton a classfcaton problem as to tpes of varables e.g. X - vector of observatons features n te orld Y - state class of te orld X R 2 fever blood

More information