The Karush-Kuhn-Tucker. Nuno Vasconcelos ECE Department, UCSD
|
|
- Kerry Dawson
- 6 years ago
- Views:
Transcription
1 e Karus-Kun-ucker condtons and dualt Nuno Vasconcelos ECE Department, UCSD
2 Optmzaton goal: nd mamum or mnmum o a uncton Denton: gven unctons, g, 1,...,k and, 1,...m dened on some doman Ω R n mn w, w Ω subect to g w w,, or compactness we wrte gw nstead o g w,. Smlarl w we derved necessar and sucent or local optmalt n te absence o constrants equalt constrants onl
3 Mnma condtons unconstraned let w be contnuousl derentable w s a local mnmum o w and onl w s a local mnmum o w and onl as zero gradent at w w and te Hessan o at w s postve dente w n t d d w d R were d d w d R, 1 n M L n n L
4 Mama condtons unconstraned let w be contnuousl derentable w s a local mamum o w and onl w s a local mamum o w and onl as zero gradent at w w and te Hessan o at w s negatve dente w n t d d w d R were d d w d R, 1 n M L n n L
5 Constraned optmzaton wt equalt constrants onl eorem: consder te problem arg mn subect to were te constrant gradents are lnearl ndependent. en s a soluton and onl tere ets a unque vector λ, suc tat m λ 1 + λ + m 1 λ, s.t. 5
6 Alternatve ormulaton state te condtons troug te Lagrangan m t t b tl tt, 1 L m + λ λ te teorem can be compactl wrtten as, L λ,, L L L st L, λ λ λ λ λ te entres o λ are reerred to as Lagrange multplers,, L s.t. λ 6
7 Geometrc nterpretaton dervatve o along d s lm α w + αd α w d w d. w.cos d, w ts means tatt greatest ncrease wen d no ncrease wen d snce tere s no ncrease wen d s tangent to so-contour k te gradent s perpendcular to te tangent o te so-contour allows geometrc nterpretaton o te Lagrangan g condtons no ncrease 7
8 Lagrangan optmzaton geometrc nterpretaton: snce s a so-contour o, s perpendcular to te so-contour sas tat span{ }.e. to tangent space o te constrant surace ntutve drecton o largest ncrease o s to constrant surace te gradent s zero along te constrant no wa to gve an nntesmal gradent step, wtout endng up volatng t t s mpossble to ncrease and stll sats te constrant span{} tg plane 8
9 Eample consder te problem mn 1 + subect to to te so-contours o 1 + k
10 Eample consder te problem mn 1 + subect to 1 + to te so-contour o
11 Eample recall tat dervatve along d s w + +α d w lm d. w.cos d, w α α crtcal pont crtcal pont - movng along te tangent s descent as long as cos tg, < -.e. π/ < angle,tg < 3π/ - can alwas nd suc d unless tg - crtcal pont wen - to nd wc tpe we need nd order as beore 11
12 Alternatve vew consder te tangent space to te so-contour ts s te subspace o rst order easble varatons { } V, space o or wc + satses te constrant up to rst order appromaton V easble varatons 1
13 Feasble varatons multplng our rst Lagrangan condton b + λ t ollows tat m 1, V ts s a generalzaton o n unconstraned case mples tat V and tereore note: Hessan constrant onl dened or n V makes sense: we cannot move anwere else, does not reall matter wat Hessan s outsde V 13
14 Inequalt constrants wat appens wen we ntroduce nequaltes? arg mn subect to, g we start b denng te set o actve nequalt constrants { g } A or eample 1, 1 +, ,
15 Actve nequalt constrants we ave a mnmum at, <, <, and 5 < arenactve - 5 s actve -5 note tat a local mnmum or ts problem would stll be a local mnmum we removed te nnactve constrants nnactve constrants do not do antng actve constrants are equaltes nnactve actve 15
16 Constraned optmzaton ence, te problem arg mn subect to, g s equvalent to arg mn subect to, g, A ts s a problem wt equalt constrants, tere must be a λ and µ, A, suc tat + m 1 λ + µ g A wc does not cange we assgn a zero Lagrange g multpler to te nnactve constrants 16
17 Constraned optmzaton lettng µ, A, + m 1 r λ + µ g 1 tere s one nal constrant t wc s µ, or all due to te ollowng pcture g as to pont nward oterwse we would ave a mamum o g as to pont outward oterwse g would ncrease nward,.e. g would be non-negatve g nsde wen we put togeter all tese constrants we obtan te amed Karus-Kun-ucker Kun KK condtons 17
18 e KK condtons eorem: or te problem, arg mn g to subect s a local mnmum and onl tere est λ and µ suc tat 1 1 g r m + + µ λ,, v A r m µ µ { },,, 1 1 A g V were V g v r m + + and µ λ 18 { },, A g V were and
19 Geometrc nterpretaton Let s orget te equalt constrants or now later we wll see tat te do not cange muc consder te problem arg mn subect to g rom te KK condtons, te soluton satses µ [ L, µ ], µ, A wt L, µ r + 1 µ g 19
20 Geometrc nterpretaton wc s equvalent to [ ] g [ L, µ ] mn + µ L mn µ wt we tus ave µ, and µ, A + µ g L + µ g L or w z - b w z - b were 1 b L, w, z µ g plane n z-space normal w, bas b L s n al-space ponted to b w
21 Geometrc Interpretaton rom b we ave L, 1 w, z µ g snce µ, w s alwas n te rst quadrant snce rst coordnate s 1, w s never parallel to g as ts can be vsualzed n z-space space as also, two cases: 1 g w z - b L te -ntercept s,l, s te mnmum o L w admssble planes w R g,, g R r-1 1
22 Geometrc Interpretaton b L, 1 w, µ case g< z g te constrants are nnactve µ w 1, plane s orzontal w z - b L te -ntercept s,l, s te mnmum o L.e. ts s alwas te case n general m o actve and nnactve but beavor s ts w g, g< R, g R r-1
23 In summar L mn wt µ [ g ] [ L, µ ] mn + µ s equvalent to, and µ, A w z - b 1 b L, w w, z z - b µ g can be vsualzed as g R g< w R easble g,, w g,, g R r-1 g< g R r-1 3
24 Dualt [ g ] [ L, µ ] mn + L mn µ wt µ, and µ, A does not appear terrbl dcult once I know µ ow I do nd ts value? consder ts uncton or an µ [ L, µ ] mn[ g ] q µ mn + µ wt µ ts s equvalent to w z - b b q µ, w w z - b 1, z µ g te pcture s te same wt L replaced b qµ 4
25 Dualt notng tat perplane w,b stll as to support easble set and we stll ave µ> ts leads to w g R g< w easble w R g,, w,,qµ g R r-1 g< g R r-1,qµ 5
26 Dualt note tat qµ L we keep ncreasng qµ we wll get qµ L we cannot go beond L would move to g > ts s eactl te denton o te dual problem ma q µ µ note: [ L, µ ] mn[ g ] q µ mn + µ wt µ qµ ma go to - or some µ, wc means tat tere s no Lagrange multpler plane would be vertcal. ts s avoded b ntroducng te constrant { µ µ > } µ D q D q 6
27 Dualt we tereore ave a two step recpe to nd te optmal soluton 1.or an µ, solve [ L, µ ] mn[ µ g ] q µ mn +.ten solve ma q µ µ, µ Dq one o te reasons w ts s nterestng s tat te second problem turns out to be qute manageable 7
28 Dualt eorem: D q s a conve set and q s concave on D q Proo: or an, µ, µ and α [,1] L + 1 α µ + αµ + 1 α µ, αµ g + αµ g + 1 α µ g [ µ g ] 1 α [ µ g ] α α L, µ + 1 α L, µ and takng mn on bot sdes mn L we ave, αµ + 1 α µ mn[ αl, µ + 1 α L, µ ] α mn L, µ + 1 α mn L, µ αµ + 1 α µ αq µ + 1 α q µ q 8
29 Dualt we ave αµ + 1 α µ αq µ + 1 α q µ q rom wc two conclusons ollow µ D q and µ D q qµ>-, qµ>- αµ+ 1-αµ D q Hence D q s sconve b denton o concavt mples tat q s concave over D q ts s a ver appealng result, snce conve optmzaton problems are among te easest to solve note tat te dual s alwas concave, rrespectve o te prmal problem te ollowng result onl proves wat we alread ave nerred rom te grapcal nterpretaton 9
30 Dualt eorem: weak dualt t s alwas true tat Proo: q or an µ, and wt g, q µ mn L z, µ + µ g z snce µ g. Hence q ma q µ µ and snce ts olds or an q mn, g 3
31 Dualt gap q we sa tat tere s no dualt gap. Oterwse tere s a dualt gap. te DG constrans te estence o Lagrange multplers eorem: tere s no dualt gap, te set o Lagrange multplers s te set o optmal dual solutons tere s a dualt gap, tere are no Lagrange multplers l Proo: b denton, µ µ s a Lagrange multpler and onl q µ q wc, rom te prevous teorem, olds and onl q,.e. tere s no dualt gap 31
32 Dualt gap note tat tere are stuatons n wc te dual problem as a soluton, but or wc tere s no Lagrange g multpler. or eample: ts s a vald dual problem owever te constrant µ g s not satsed and R, q g < g R r-1 n summar, dualt s nterestng onl wen tere s no dualt gap,q 3
33 33
Optimization. Nuno Vasconcelos ECE Department, UCSD
Optmzaton Nuno Vasconcelos ECE Department, UCSD Optmzaton many engneerng problems bol on to optmzaton goal: n mamum or mnmum o a uncton Denton: gven unctons, g,,...,k an h,,...m ene on some oman Ω R n
More informationLinear discriminants. Nuno Vasconcelos ECE Department, UCSD
Lnear dscrmnants Nuno Vasconcelos ECE Department UCSD Classfcaton a classfcaton problem as to tpes of varables e.g. X - vector of observatons features n te orld Y - state class of te orld X R 2 fever blood
More informationShuai Dong. Isaac Newton. Gottfried Leibniz
Computatonal pyscs Sua Dong Isaac Newton Gottred Lebnz Numercal calculus poston dervatve ntegral v velocty dervatve ntegral a acceleraton Numercal calculus Numercal derentaton Numercal ntegraton Roots
More informationMathematical Economics MEMF e ME. Filomena Garcia. Topic 2 Calculus
Mathematcal Economcs MEMF e ME Flomena Garca Topc 2 Calculus Mathematcal Economcs - www.seg.utl.pt/~garca/economa_matematca . Unvarate Calculus Calculus Functons : X Y y ( gves or each element X one element
More informationOPTIMISATION. Introduction Single Variable Unconstrained Optimisation Multivariable Unconstrained Optimisation Linear Programming
OPTIMIATION Introducton ngle Varable Unconstraned Optmsaton Multvarable Unconstraned Optmsaton Lnear Programmng Chapter Optmsaton /. Introducton In an engneerng analss, sometmes etremtes, ether mnmum or
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationORDINARY DIFFERENTIAL EQUATIONS EULER S METHOD
Numercal Analss or Engneers German Jordanan Unverst ORDINARY DIFFERENTIAL EQUATIONS We wll eplore several metods o solvng rst order ordnar derental equatons (ODEs and we wll sow ow tese metods can be appled
More information, rst we solve te PDE's L ad L ad n g g (x) = ; = ; ; ; n () (x) = () Ten, we nd te uncton (x), te lnearzng eedbac and coordnates transormaton are gve
Freedom n Coordnates Transormaton or Exact Lnearzaton and ts Applcaton to Transent Beavor Improvement Kenj Fujmoto and Tosaru Suge Dvson o Appled Systems Scence, Kyoto Unversty, Uj, Kyoto, Japan suge@robotuassyoto-uacjp
More informationSolutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.
Solutons HW #2 Dual of general LP. Fnd the dual functon of the LP mnmze subject to c T x Gx h Ax = b. Gve the dual problem, and make the mplct equalty constrants explct. Soluton. 1. The Lagrangan s L(x,
More informationFinite Difference Method
7/0/07 Instructor r. Ramond Rump (9) 747 698 rcrump@utep.edu EE 337 Computatonal Electromagnetcs (CEM) Lecture #0 Fnte erence Method Lecture 0 These notes ma contan coprghted materal obtaned under ar use
More informationCOS 521: Advanced Algorithms Game Theory and Linear Programming
COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton
More informationOn a nonlinear compactness lemma in L p (0, T ; B).
On a nonlnear compactness lemma n L p (, T ; B). Emmanuel Matre Laboratore de Matématques et Applcatons Unversté de Haute-Alsace 4, rue des Frères Lumère 6893 Mulouse E.Matre@ua.fr 3t February 22 Abstract
More informationLagrange Multipliers Kernel Trick
Lagrange Multplers Kernel Trck Ncholas Ruozz Unversty of Texas at Dallas Based roughly on the sldes of Davd Sontag General Optmzaton A mathematcal detour, we ll come back to SVMs soon! subject to: f x
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationPhysics 2A Chapter 3 HW Solutions
Phscs A Chapter 3 HW Solutons Chapter 3 Conceptual Queston: 4, 6, 8, Problems: 5,, 8, 7, 3, 44, 46, 69, 70, 73 Q3.4. Reason: (a) C = A+ B onl A and B are n the same drecton. Sze does not matter. (b) C
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationComplex Variables. Chapter 18 Integration in the Complex Plane. March 12, 2013 Lecturer: Shih-Yuan Chen
omplex Varables hapter 8 Integraton n the omplex Plane March, Lecturer: Shh-Yuan hen Except where otherwse noted, content s lcensed under a BY-N-SA. TW Lcense. ontents ontour ntegrals auchy-goursat theorem
More informationA Simple Research of Divisor Graphs
The 29th Workshop on Combnatoral Mathematcs and Computaton Theory A Smple Research o Dvsor Graphs Yu-png Tsao General Educaton Center Chna Unversty o Technology Tape Tawan yp-tsao@cuteedutw Tape Tawan
More informationEE 330 Lecture 24. Small Signal Analysis Small Signal Analysis of BJT Amplifier
EE 0 Lecture 4 Small Sgnal Analss Small Sgnal Analss o BJT Ampler Eam Frda March 9 Eam Frda Aprl Revew Sesson or Eam : 6:00 p.m. on Thursda March 8 n Room Sweene 6 Revew rom Last Lecture Comparson o Gans
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationOn Pfaff s solution of the Pfaff problem
Zur Pfaff scen Lösung des Pfaff scen Probles Mat. Ann. 7 (880) 53-530. On Pfaff s soluton of te Pfaff proble By A. MAYER n Lepzg Translated by D. H. Delpenc Te way tat Pfaff adopted for te ntegraton of
More informationCHAPTER-5 INFORMATION MEASURE OF FUZZY MATRIX AND FUZZY BINARY RELATION
CAPTER- INFORMATION MEASURE OF FUZZY MATRI AN FUZZY BINARY RELATION Introducton The basc concept of the fuzz matr theor s ver smple and can be appled to socal and natural stuatons A branch of fuzz matr
More informationCENTROID (AĞIRLIK MERKEZİ )
CENTOD (ĞLK MEKEZİ ) centrod s a geometrcal concept arsng from parallel forces. Tus, onl parallel forces possess a centrod. Centrod s tougt of as te pont were te wole wegt of a pscal od or sstem of partcles
More informationCENTROID (AĞIRLIK MERKEZİ )
CENTOD (ĞLK MEKEZİ ) centrod s a geometrcal concept arsng from parallel forces. Tus, onl parallel forces possess a centrod. Centrod s tougt of as te pont were te wole wegt of a pscal od or sstem of partcles
More information5 The Laplace Equation in a convex polygon
5 Te Laplace Equaton n a convex polygon Te most mportant ellptc PDEs are te Laplace, te modfed Helmoltz and te Helmoltz equatons. Te Laplace equaton s u xx + u yy =. (5.) Te real and magnary parts of an
More informationProblem Set 4: Sketch of Solutions
Problem Set 4: Sketc of Solutons Informaton Economcs (Ec 55) George Georgads Due n class or by e-mal to quel@bu.edu at :30, Monday, December 8 Problem. Screenng A monopolst can produce a good n dfferent
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationWork is the change in energy of a system (neglecting heat transfer). To examine what could
Work Work s the change n energy o a system (neglectng heat transer). To eamne what could cause work, let s look at the dmensons o energy: L ML E M L F L so T T dmensonally energy s equal to a orce tmes
More informationCHAPTER 6 CONSTRAINED OPTIMIZATION 1: K-T CONDITIONS
Chapter 6: Constraned Optzaton CHAPER 6 CONSRAINED OPIMIZAION : K- CONDIIONS Introducton We now begn our dscusson of gradent-based constraned optzaton. Recall that n Chapter 3 we looked at gradent-based
More informationParameter estimation class 5
Parameter estmaton class 5 Multple Ve Geometr Comp 9-89 Marc Pollefes Content Background: Projectve geometr (D, 3D), Parameter estmaton, Algortm evaluaton. Sngle Ve: Camera model, Calbraton, Sngle Ve Geometr.
More informationSingle Variable Optimization
8/4/07 Course Instructor Dr. Raymond C. Rump Oce: A 337 Phone: (95) 747 6958 E Mal: rcrump@utep.edu Topc 8b Sngle Varable Optmzaton EE 4386/530 Computatonal Methods n EE Outlne Mathematcal Prelmnares Sngle
More informationCHAPTER 7 CONSTRAINED OPTIMIZATION 1: THE KARUSH-KUHN-TUCKER CONDITIONS
CHAPER 7 CONSRAINED OPIMIZAION : HE KARUSH-KUHN-UCKER CONDIIONS 7. Introducton We now begn our dscusson of gradent-based constraned optzaton. Recall that n Chapter 3 we looked at gradent-based unconstraned
More information14 Lagrange Multipliers
Lagrange Multplers 14 Lagrange Multplers The Method of Lagrange Multplers s a powerful technque for constraned optmzaton. Whle t has applcatons far beyond machne learnng t was orgnally developed to solve
More informationSolutions to Homework 7, Mathematics 1. 1 x. (arccos x) (arccos x) 1
Solutons to Homework 7, Mathematcs 1 Problem 1: a Prove that arccos 1 1 for 1, 1. b* Startng from the defnton of the dervatve, prove that arccos + 1, arccos 1. Hnt: For arccos arccos π + 1, the defnton
More informationLecture 2 Solution of Nonlinear Equations ( Root Finding Problems )
Lecture Soluton o Nonlnear Equatons Root Fndng Problems Dentons Classcaton o Methods Analytcal Solutons Graphcal Methods Numercal Methods Bracketng Methods Open Methods Convergence Notatons Root Fndng
More informationSection 15.6 Directional Derivatives and the Gradient Vector
Section 15.6 Directional Derivatives and te Gradient Vector Finding rates of cange in different directions Recall tat wen we first started considering derivatives of functions of more tan one variable,
More informationCompetitive Experimentation and Private Information
Compettve Expermentaton an Prvate Informaton Guseppe Moscarn an Francesco Squntan Omtte Analyss not Submtte for Publcaton Dervatons for te Gamma-Exponental Moel Dervaton of expecte azar rates. By Bayes
More informationThe Finite Element Method: A Short Introduction
Te Fnte Element Metod: A Sort ntroducton Wat s FEM? Te Fnte Element Metod (FEM) ntroduced by engneers n late 50 s and 60 s s a numercal tecnque for solvng problems wc are descrbed by Ordnary Dfferental
More informationMachine Learning. What is a good Decision Boundary? Support Vector Machines
Machne Learnng 0-70/5 70/5-78 78 Sprng 200 Support Vector Machnes Erc Xng Lecture 7 March 5 200 Readng: Chap. 6&7 C.B book and lsted papers Erc Xng @ CMU 2006-200 What s a good Decson Boundar? Consder
More informationCHAPTER 7 CONSTRAINED OPTIMIZATION 2: SQP AND GRG
Chapter 7: Constraned Optmzaton CHAPER 7 CONSRAINED OPIMIZAION : SQP AND GRG Introducton In the prevous chapter we eamned the necessary and suffcent condtons for a constraned optmum. We dd not, however,
More informationLogarithmic functions
Roberto s Notes on Differential Calculus Capter 5: Derivatives of transcendental functions Section Derivatives of Logaritmic functions Wat ou need to know alread: Definition of derivative and all basic
More informationPractical Newton s Method
Practcal Newton s Method Lecture- n Newton s Method n Pure Newton s method converges radly once t s close to. It may not converge rom the remote startng ont he search drecton to be a descent drecton rue
More informationStanford University CS254: Computational Complexity Notes 7 Luca Trevisan January 29, Notes for Lecture 7
Stanford Unversty CS54: Computatonal Complexty Notes 7 Luca Trevsan January 9, 014 Notes for Lecture 7 1 Approxmate Countng wt an N oracle We complete te proof of te followng result: Teorem 1 For every
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More informationMaximum Likelihood Estimation (MLE)
Maxmum Lkelhood Estmaton (MLE) Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Wnter 01 UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y
More informationPattern Classification
Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher
More informationNot-for-Publication Appendix to Optimal Asymptotic Least Aquares Estimation in a Singular Set-up
Not-for-Publcaton Aendx to Otmal Asymtotc Least Aquares Estmaton n a Sngular Set-u Antono Dez de los Ros Bank of Canada dezbankofcanada.ca December 214 A Proof of Proostons A.1 Proof of Prooston 1 Ts roof
More informationDegrees of Freedom. Spherical (ball & socket) 3 (3 rotation) Two-Angle (universal) 2 (2 rotation)
ME 6590 Multbody Dynamcs Connectn Jonts Part I o Connectn jonts constran te relatve moton between adjonn bodes n a multbody system. Jonts rane rom allown no relatve moton (a rd jont) to allown all motons
More informationEndogenous timing in a mixed oligopoly consisting of a single public firm and foreign competitors. Abstract
Endogenous tmng n a mxed olgopoly consstng o a sngle publc rm and oregn compettors Yuanzhu Lu Chna Economcs and Management Academy, Central Unversty o Fnance and Economcs Abstract We nvestgate endogenous
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationChapter 3 Differentiation and Integration
MEE07 Computer Modelng Technques n Engneerng Chapter Derentaton and Integraton Reerence: An Introducton to Numercal Computatons, nd edton, S. yakowtz and F. zdarovsky, Mawell/Macmllan, 990. Derentaton
More informationExercises of Fundamentals of Chemical Processes
Department of Energ Poltecnco d Mlano a Lambruschn 4 2056 MILANO Exercses of undamentals of Chemcal Processes Prof. Ganpero Gropp Exercse 7 ) Estmaton of the composton of the streams at the ext of an sothermal
More informationConvex Optimization. Optimality conditions. (EE227BT: UC Berkeley) Lecture 9 (Optimality; Conic duality) 9/25/14. Laurent El Ghaoui.
Convex Optmzaton (EE227BT: UC Berkeley) Lecture 9 (Optmalty; Conc dualty) 9/25/14 Laurent El Ghaou Organsatonal Mdterm: 10/7/14 (1.5 hours, n class, double-sded cheat sheet allowed) Project: Intal proposal
More informationImage classification. Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing i them?
Image classfcaton Gven te bag-of-features representatons of mages from dfferent classes ow do we learn a model for dstngusng tem? Classfers Learn a decson rule assgnng bag-offeatures representatons of
More informationLecture 10 Support Vector Machines. Oct
Lecture 10 Support Vector Machnes Oct - 20-2008 Lnear Separators Whch of the lnear separators s optmal? Concept of Margn Recall that n Perceptron, we learned that the convergence rate of the Perceptron
More informationQuantum Mechanics I - Session 4
Quantum Mechancs I - Sesson 4 Aprl 3, 05 Contents Operators Change of Bass 4 3 Egenvectors and Egenvalues 5 3. Denton....................................... 5 3. Rotaton n D....................................
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationExponentials and Logarithms Review Part 2: Exponentials
Eponentials and Logaritms Review Part : Eponentials Notice te difference etween te functions: g( ) and f ( ) In te function g( ), te variale is te ase and te eponent is a constant. Tis is called a power
More informationMachine Learning. Support Vector Machines. Eric Xing , Fall Lecture 9, October 6, 2015
Machne Learnng 0-70 Fall 205 Support Vector Machnes Erc Xng Lecture 9 Octoer 6 205 Readng: Chap. 6&7 C.B ook and lsted papers Erc Xng @ CMU 2006-205 What s a good Decson Boundar? Consder a nar classfcaton
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More information2.8 The Derivative as a Function
.8 Te Derivative as a Function Typically, we can find te derivative of a function f at many points of its domain: Definition. Suppose tat f is a function wic is differentiable at every point of an open
More informationBayesian decision theory. Nuno Vasconcelos ECE Department, UCSD
Bayesan decson theory Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world observatons decson functon L[,y] loss of predctn y wth the epected value of the
More informationESCUELA LATINOAMERICANA DE COOPERACIÓN Y DESARROLLO Especialización en Cooperación Internacional para el Desarrollo
ESCUELA LATINOAMERICANA DE COOPERACIÓN Y DESARROLLO Especialización en Cooperación Internacional para el Desarrollo A SURVIVAL KIT IN CASE O.MATHEMATICS b Marco Missaglia * Universit of Pavia September
More informationPhysics 2A Chapters 6 - Work & Energy Fall 2017
Physcs A Chapters 6 - Work & Energy Fall 017 These notes are eght pages. A quck summary: The work-energy theorem s a combnaton o Chap and Chap 4 equatons. Work s dened as the product o the orce actng on
More informationChapter Eight. Review and Summary. Two methods in solid mechanics ---- vectorial methods and energy methods or variational methods
Chapter Eght Energy Method 8. Introducton 8. Stran energy expressons 8.3 Prncpal of statonary potental energy; several degrees of freedom ------ Castglano s frst theorem ---- Examples 8.4 Prncpal of statonary
More informationSummary with Examples for Root finding Methods -Bisection -Newton Raphson -Secant
Summary wth Eamples or Root ndng Methods -Bsecton -Newton Raphson -Secant Nonlnear Equaton Solvers Bracketng Graphcal Open Methods Bsecton False Poston (Regula-Fals) Newton Raphson Secant All Iteratve
More informationWhat is LP? LP is an optimization technique that allocates limited resources among competing activities in the best possible manner.
(C) 998 Gerald B Sheblé, all rghts reserved Lnear Prograng Introducton Contents I. What s LP? II. LP Theor III. The Splex Method IV. Refneents to the Splex Method What s LP? LP s an optzaton technque that
More information( ) 2 ( ) ( ) Problem Set 4 Suggested Solutions. Problem 1
Problem Set 4 Suggested Solutons Problem (A) The market demand functon s the soluton to the followng utlty-maxmzaton roblem (UMP): The Lagrangean: ( x, x, x ) = + max U x, x, x x x x st.. x + x + x y x,
More information: Numerical Analysis Topic 2: Solution of Nonlinear Equations Lectures 5-11:
764: Numercal Analyss Topc : Soluton o Nonlnear Equatons Lectures 5-: UIN Malang Read Chapters 5 and 6 o the tetbook 764_Topc Lecture 5 Soluton o Nonlnear Equatons Root Fndng Problems Dentons Classcaton
More informationLecture 26 Finite Differences and Boundary Value Problems
4//3 Leture 6 Fnte erenes and Boundar Value Problems Numeral derentaton A nte derene s an appromaton o a dervatve - eample erved rom Talor seres 3 O! Negletng all terms ger tan rst order O O Tat s te orward
More informationPolynomial Barrier Method for Solving Linear Programming Problems
Internatonal Journal o Engneerng & echnology IJE-IJENS Vol: No: 45 Polynoal Barrer Method or Solvng Lnear Prograng Probles Parwad Moengn, Meber, IAENG Abstract In ths wor, we study a class o polynoal ordereven
More informationPre-Calculus Summer Assignment
Pre-Calculus Summer Assgnment Dear Future Pre-Calculus Student, Congratulatons on our successful completon of Algebra! Below ou wll fnd the summer assgnment questons. It s assumed that these concepts,
More informationPHYS 1101 Practice problem set 12, Chapter 32: 21, 22, 24, 57, 61, 83 Chapter 33: 7, 12, 32, 38, 44, 49, 76
PHYS 1101 Practce problem set 1, Chapter 3: 1,, 4, 57, 61, 83 Chapter 33: 7, 1, 3, 38, 44, 49, 76 3.1. Vsualze: Please reer to Fgure Ex3.1. Solve: Because B s n the same drecton as the ntegraton path s
More informationSELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d.
SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationGeneral Tips on How to Do Well in Physics Exams. 1. Establish a good habit in keeping track of your steps. For example, when you use the equation
General Tps on How to Do Well n Physcs Exams 1. Establsh a good habt n keepng track o your steps. For example when you use the equaton 1 1 1 + = d d to solve or d o you should rst rewrte t as 1 1 1 = d
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationMean Field / Variational Approximations
Mean Feld / Varatonal Appromatons resented by Jose Nuñez 0/24/05 Outlne Introducton Mean Feld Appromaton Structured Mean Feld Weghted Mean Feld Varatonal Methods Introducton roblem: We have dstrbuton but
More informationOne Dimensional Axial Deformations
One Dmensonal al Deformatons In ths secton, a specfc smple geometr s consdered, that of a long and thn straght component loaded n such a wa that t deforms n the aal drecton onl. The -as s taken as the
More informationCHAPTER 4d. ROOTS OF EQUATIONS
CHAPTER 4d. ROOTS OF EQUATIONS A. J. Clark School o Engneerng Department o Cvl and Envronmental Engneerng by Dr. Ibrahm A. Assakka Sprng 00 ENCE 03 - Computaton Methods n Cvl Engneerng II Department o
More informationSIMPLE LINEAR REGRESSION
Smple Lnear Regresson and Correlaton Introducton Prevousl, our attenton has been focused on one varable whch we desgnated b x. Frequentl, t s desrable to learn somethng about the relatonshp between two
More informationTR/95 February Splines G. H. BEHFOROOZ* & N. PAPAMICHAEL
TR/9 February 980 End Condtons for Interpolatory Quntc Splnes by G. H. BEHFOROOZ* & N. PAPAMICHAEL *Present address: Dept of Matematcs Unversty of Tabrz Tabrz Iran. W9609 A B S T R A C T Accurate end condtons
More informationCOMP4630: λ-calculus
COMP4630: λ-calculus 4. Standardsaton Mcael Norrs Mcael.Norrs@ncta.com.au Canberra Researc Lab., NICTA Semester 2, 2015 Last Tme Confluence Te property tat dvergent evaluatons can rejon one anoter Proof
More information1. Fundamentals 1.1 Probability Theory Sample Space and Probability Random Variables Limit Theories
1. undamentals 1.1 robablt Theor 1.1.1 Sample Space and robablt 1.1.2 Random Varables 1.1.3 Lmt Theores 1.1 robablt Theor A statstcal model probablt model deals wth eperment whose outcome s not precsel
More informationThe Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor
Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T
More informationContinuity and Differentiability Worksheet
Continuity and Differentiability Workseet (Be sure tat you can also do te grapical eercises from te tet- Tese were not included below! Typical problems are like problems -3, p. 6; -3, p. 7; 33-34, p. 7;
More information( ) [ ( k) ( k) ( x) ( ) ( ) ( ) [ ] ξ [ ] [ ] [ ] ( )( ) i ( ) ( )( ) 2! ( ) = ( ) 3 Interpolation. Polynomial Approximation.
3 Interpolaton {( y } Gven:,,,,,, [ ] Fnd: y for some Mn, Ma Polynomal Appromaton Theorem (Weerstrass Appromaton Theorem --- estence ε [ ab] f( P( , then there ests a polynomal
More informationMaximum Likelihood Estimation
Maxmum Lkelhood Estmaton INFO-2301: Quanttatve Reasonng 2 Mchael Paul and Jordan Boyd-Graber MARCH 7, 2017 INFO-2301: Quanttatve Reasonng 2 Paul and Boyd-Graber Maxmum Lkelhood Estmaton 1 of 9 Why MLE?
More informationDIFFERENTIAL SCHEMES
DIFFERENTIAL SCHEMES RAYMOND T. HOOBLER Dedcated to the memory o Jerry Kovacc 1. schemes All rngs contan Q and are commutatve. We x a d erental rng A throughout ths secton. 1.1. The topologcal space. Let
More informationNeural networks. Nuno Vasconcelos ECE Department, UCSD
Neural networs Nuno Vasconcelos ECE Department, UCSD Classfcaton a classfcaton problem has two types of varables e.g. X - vector of observatons (features) n the world Y - state (class) of the world x X
More informationClassical Mechanics Virtual Work & d Alembert s Principle
Classcal Mechancs Vrtual Work & d Alembert s Prncple Dpan Kumar Ghosh UM-DAE Centre for Excellence n Basc Scences Kalna, Mumba 400098 August 15, 2016 1 Constrants Moton of a system of partcles s often
More informationSpring Force and Power
Lecture 13 Chapter 9 Sprng Force and Power Yeah, energy s better than orces. What s net? Course webste: http://aculty.uml.edu/andry_danylov/teachng/physcsi IN THIS CHAPTER, you wll learn how to solve problems
More informationNumerical Differentiation
Part 5 Capter 19 Numercal Derentaton PowerPonts organzed by Dr. Mcael R. Gustason II, Duke Unversty Revsed by Pro. Jang, CAU All mages copyrgt Te McGraw-Hll Companes, Inc. Permsson requred or reproducton
More informationf(x,y) = (4(x 2 4)x,2y) = 0 H(x,y) =
Problem Set 3: Unconstraned mzaton n R N. () Fnd all crtcal ponts of f(x,y) (x 4) +y and show whch are ma and whch are mnma. () Fnd all crtcal ponts of f(x,y) (y x ) x and show whch are ma and whch are
More informationMath 324 Advanced Financial Mathematics Spring 2008 Final Exam Solutions May 2, 2008
Mat 324 Advanced Fnancal Matematcs Sprng 28 Fnal Exam Solutons May 2, 28 Ts s an open book take-ome exam. You may work wt textbooks and notes but do not consult any oter person. Sow all of your work and
More information