The Bellman Equation
|
|
- Kristina Lang
- 5 years ago
- Views:
Transcription
1 The Bellman Eqaton Reza Shadmehr In ths docment I wll rovde an elanaton of the Bellman eqaton, whch s a method for otmzng a cost fncton and arrvng at a control olcy.. Eamle of a game Sose that or states refer to the oston on a grd, as shown below. If we are at the goal state, then the state cost er tme ste s zero. If we are at any other state, the state cost er tme ste s 5. Let s se the term J to refer to ths state cost er tme ste: J ( The goal state s at row, col., whch means that f we are at ths state, we ncr no state costs. The doble lnes refer to a wall, reventng one to move from one state to the neghborng state. That s, there s a wall between the to left and to mddle states. If we erform some acton (say, move from one bo to the neghborng bo, there wll be a motor cost er tme ste, whch we refer to wth symbol J. The motor cost s one f we move J, and zero otherwse. So the total cost er tme ste s: ( n ( n J J α ( ( The term ( n ( n refers to the olcy that we have. Ths olcy secfes the acton that we wll erform for each state at tme ont n. For eamle, f we ck a random olcy, then we mght have actons that look lke ths: ( n (3 Sose or fnal tme ste s. If we are now at tme ont k, or objectve s to fnd the olcy that ( mnmzes the total cost to go α. Let s defne the goodness of each olcy va a vale fncton: k
2 ( ( k ( k ( ( k α (4 If we are at the last tme ste, then the vale of or olcy s smly the cost er ste at ths last tme ste: ( J J (5 The otmm olcy s one that mnmzes Eq. (4, whch s smly to do nothng: * ( ( (6 In that case, we have: ( (7 * J The Bellman otmalty rncle states that an otmal olcy has the roerty that whatever the ntal state and ntal decson are, the remanng decsons mst consttte an otmal olcy wth regard to the state resltng from the frst decson. Ths means that n order to fnd the otmal olcy for tme ont, for each state we mst fnd the command that mnmzes the followng: ( ( ( { α * ( } * argmn (8 Once we have the otmal command for each state, the vale of that state s: ( mn ( ( α { } * Consder the to mddle state. If we were to rodce an acton that moves s down, we wold have the ( state cost of 5, and motor cost of, and so α 6. The vale of the state we get to s zero. So the ( total vale of ths acton s 6. If we were to stay and not move, α 5 ls the vale of the state that we get to (the crrent state, whch s 5, for a sm total of 0. So the vale of the acton of movng down s 6, whereas the acton of dong nothng has the vale of 0. The vale of the acton of movng to the rght s. So the best acton that we can do for the to mddle state s to move down. Smlarly, the best acton that we can do for the bottom mddle state s to stay stll. In ths way, we can defne the acton that mnmzes Eq. (8 for each state, resltng n or olcy for tme ste :
3 ( * ( (9 The vale of ths otmal olcy s: ( ( ( Before we roceed to the net ste, t s worthwhle to take another look at Eq. (0. We have a vale assocated wth each state. Ths vale s the total cost that wll be ncrred f, startng at a gven state, we were to erform the best seqence of actons ossble. The seqence of actons wll rodce a seqence of cost er stes that together sm to be the vale assgned to each state. In a sense, when we fnd orselves at a gven state, the vale of that state reresents the lowest cost to go that we hoe to ncr f we were to rodce the best actons ossble. Now we reeat ths rocess for tme ste. Let s reconsder the to mddle state. The vale of stayng stll s: J ( J ( * The vale of movng down s The vale of movng to the rght s. The best acton remans to move down. Consder the bottom mddle state. The vale of movng s. The vale of stayng stll s 5. The best acton s to move (or move to rght to neghbor. The otmal olcy at tme ste s: ( * ( The vale of ths otmal olcy s: ( ( ( We comte the otmal olcy for tme ste 3 :
4 ( * ( 3 (3 The vale of ths otmal olcy s: ( ( 3 ( Smlarly, we comte the otmal olcy for tme ste 4 : ( * ( 4 (5 The vale of ths otmal olcy s: ( ( 4 ( Now the nterestng reslt comes when we consder tme ste 5. Consder the to left bo. If we were to move down, the cost s 30. If we were to stay, the cost s also 30. So the otmal olcy s ether to stay stll or move down, as both gves s the same cost. The reason for ths s that the effort cost of movng s so large for ths state (t s so far from the goal state that the reward of gettng to the goal only barely comensates for the cost of movng. Sose we decde to stay, and so we have: The vale of ths otmal olcy s: ( ( * ( 5 * ( 4 ( ( ( 5 ( However, at tme ste 6, for the to left state or otmal acton s to move down, and so we get:
5 ( * ( 6 (9 For each tme ste, we have sed the Bellman eqaton (Eq. 8 to fnd the otmal feedback control olcy.. Eamle of a lnear system wthot nose In condtons for whch we are dealng wth a lnear dynamcal system, the vale fncton wll trn ot to be a qadratc fncton of state, and the control olcy wll become a lnear fncton of state. These class of control roblems are also called Lnear Qadratc Reglators. Let s start wth a lnear system wthot nose. ( n ( n ( n A B y ( n ( n C (0 We have the followng cost er ste: ( n ( n T ( n ( n ( n T ( n α y T y L ( nt T ( n ( n ( nt ( n C T C L ( Let s begn at the fnal tme ste n (. At ths tme, the best acton that we can erform s one that mnmzes the cost acton at the fnal tme ont s: ( α. That acton s: ( * 0 If we erform ths otmal acton, the vale of the state we are at s: 0. That s, regardless of state, the otmal olcy of ( ( ( T T ( ( * C T C (3 We see that at the fnal tme ont, the vale fncton s a qadratc fncton of state. Let s defne ( matr W as follows: ( T ( And so the vale for the otmal olcy can be wrtten as: W C T C (4 ( ( T ( ( * W (5
6 In order to fnd the otmal olcy for tme ont, for each state we mst fnd the command that mnmzes the sm of the cost at the crrent tme ste, ls the vale of the state that we arrve at after we rodce the command: ( arg mn { * ( α, } * ( ( ( ( (6 We can wrte the eresson nsde the brackets as: ( ( T T ( ( ( T ( α C T C L T ( ( ( ( ( ( ( ( (, ( ( A B W A B ( T T ( ( ( T T ( ( AW A BW B ( T T ( ( BW A (7 ( To mnmze the sm n Eq. (6, we fnd ts dervatve wth resect to and set t eqal to zero. Ths gves s the otmal commands: Let s defne the followng matr: ( T ( ( T ( ( L B W B B W A 0 (8 We now can wrte the otmal olcy as follows: *( T ( T ( ( L B W B B W A (9 ( T ( T ( G L B W B B W A (3 * ( ( ( G The vale of each state nder the otmal olcy can be wrtten as (3 ( ( ( ( *( ( (, (, *( α * ( T T ( ( ( T ( T ( ( BT B G LG AW A G BW BG ( T ( T T ( ( G B W A ( T T ( ( ( T ( T T ( ( ( Notce that the vale fncton s a qadratc fncton of state. We can smlfy t a lttle sng the defnton of ( G : ( T ( ( T T ( ( ( T T ( ( G LG G B W BG G L B W B G ( T T ( G B W A (33 ( Let s defne W as follows: ( T ( T ( ( T T ( ( T T ( W BT B AW A G BW A G BW A T ( T ( ( T T ( BT B AW AG BW A (34
7 We can wrte the vale fncton as: We now have a rece. For ste we have: ( ( ( ( W (35 ( T ( T ( G L B W B B W A (36 And the followng vale fncton: ( ( ( ( W ( T ( T ( ( T T ( W B T B A W AG B W A (37 ( And so the rocedre s as follows: startng from the last tme ont, we comte G (whch s zero ( ( and W (Eq. 4. We net move tme ont and comte G ( (Eq. 3 and W (Eq. 34. ( ( We then se Eq. (36 to comte G and W (Eq. 37. And so on, ntl we reach tme ont 0. For each tme ste, we wll have a olcy that transforms or crrent state nto a motor command. As an eamle, let s consder movng a sngle jont model of the elbow. The state of the system s defned by ts oston and velocty (referrng to anglar oston and velocty. The dynamcs of the system are descrbed as follows: k b 0 m m m Ac Cc y 0 The above eqatons are wrtten n contnos tme. To reresent t n dscrete tme (wth a tme ste of Δ t, we can wrte the dscrete eqatons as follows: ( k ( k ( k A B ( k ( k y C A ( I Ac Δt B BcΔt C C c I wanted the elbow to make a movement that ended at a goal state of t ( 300 ms 0.5, wth zero velocty, and held there for an addtonal 00ms. I sed the followng arameter vales for the arm:
8 k 3 N. m / rad b 0.45 N. ms. / rad m 0.3 kgm. / rad I set the state cost matr T to have the followng vales as a fncton of tme: 3. Eamle of a lnear system wth sgnal deendent nose Let s consder a smle scalar system of the form: ( t ( t ( t ( t ( t ( t a b( ε ε N 0, c ( ( t ( t y ε ε N 0, σ y y y In ths system, the state s a scalar, and so s the observaton y. However, notce that n ths system the nose s sgnal deendent. That s, the varance of the nose deends on the sze of the motor ( t commands. We begn by eressng the random varable ε n terms of random varable ( t φ N 0, ( t and : ( t ( t ( t c φ ( ( ( Let s sose that the cost er ste s: α t α( t α( t ont the otmal olcy * ( ( 0 ( ( α ε. Ths mles that at the last tme and the vale of the states acheved nder ths olcy s. We now fnd the otmal olcy for tme ste. We begn by comtng the term ( ( ( E *,. ( ( ( E ( var E ( ( ( ( E *(, αe ( ( ( α var E ( ( ( α bc ( ( a b The cost that we need to mnmze at tme ste s: ( ( ( ( ( ( ( α( α ( E *(, ( ( ( ( ( α( α( α b c ( ( a b We fnd ( that mnmzes ths cost:
9 d ( ( ( ( α ( αb c αb αab d ( ( ( α αb c αb αab g * ( ( ( 0 g However, becase s a random varable, at any tme ont we wll have an estmate of t, ˆ. And so or otmal olcy at tme ont s ( ˆ ( g ˆ (. Usng or olcy for tme ste, we can comte the vale fncton (, ˆ ( ( demonstrate that t s a qadratc fncton of, and the error n estmate of that state ( ( ˆ. ( ( ( ( ( (, ˆ ( ˆ ( ( ( ( ( αbc ( g ˆ α( a bg ˆ ( ( ( ( α αa ( ( α αb c αb ( g ˆ α α g * ( ( ( α abg ˆ ( ( ( α αa ( αabg ( ˆ zˆ zˆ z ˆ z ( ( ( ( ( ( ( (, ˆ ( α α α α ( ˆ ( ( ( ( ( w ( w ( ˆ z and ( ( ( ( αabg ˆ a abg abg ( t Now let s consder the tme ste. We can observe y and wrte the eqaton for the Kalman gan. As we wll see, the Kalman gan wll not deend on. var tt ( ˆ ( t ( tt ( tt ( t ( ( tt ˆ ˆ k y ˆ dp ( tt dk ( tt ( t k ( t ( t ( tt ( t ( t ( t εy k ˆ k k P t ( tt t ( k var( ˆ ( k ( ( σy ( t ( t ( t k k P k σy ( tt ( tt ( tt ( t ( t σy ( tt P P k k P ( tt P σ y
10 ( At tme ont t, or estmate of ˆ t s smly the ror estmate ( ˆ t ( t, and the Kalman gan. ( t ( t ˆ ˆ k y ˆ At tme ont ( tt ( tt ( tt ( t t ( t t ( t ( ˆ tt ˆ aˆ b tt aˆ ak ( y ˆ b ( t ( t ( t ( t ( t ( t ˆ aˆ ak y ˆ b ( tt ( t ( t ( ( t (. Let s wrte ˆ t n terms of we showed that the vale fncton nder the otmal olcy ( ( (, ˆ ( qadratc fncton of ( ( and the error n estmate of that state ˆ ( ( ( ( ( ( ( relatonsh as (, ˆ w ( w ( ˆ otmal olcy for tme ont.. Let s wrte that and then fnd the ( ( ( ( α( α( ( ( ( ( ( E * (, ˆ,, ˆ ( ( ( ( ( ( ( (, ˆ ( ( ( w E ( ˆ ( ( ( ( ( ( ( ( ( ( ˆ E ( a b bε aˆ ak ( y ˆ b ( ( ( ( ( E ( a( ˆ ak ( εy ˆ bε ( ( ( ( E (( a ak ( ˆ ak εy bε ( ( ( d ( a ak ( ˆ ( ( ( ( ( ( ˆ E d adk εy bd ε a ( k εy abk εyε b ε ( ( d a ( k σy b c ( ( ( ( ( ( ( ( ( ( α( α( w ( a b w b c ( ( ( ( w d a ( k σy bc ( ( d ( ( ( ( ( ( ( ( α w b c w b c w b abw d * ( ( ( ( ( ( ( ( α w b c w b c w b abw E w a b w bc E E g ( ( The best that we can do s mlement the olcy g * ( ( g ˆ. Let s show that nder ( ths olcy, the vale fncton remans qadratc n terms and the error n estmate of that state ( ( ˆ. s a
11 ( ( ( ( ( ( ( ( ( ( ( ( w d a ( k σy b c ( ( ( ( ( ( ( α w a ( w ab ( ( ( ( ( ( α bw w bc w bc ( w d a ( k ( ( ( ( ( ( ( α w a ( abw g ˆ ( ( ( ( ( abw g ( ˆ w d a ( k σ y α α w a b w b c * If we note that ˆ ˆ ( ˆ olcy as: σy z z z z, then we can wrte the vale fncton nder the otmal ( ( ( ( ( ( ( ( ( ( α ( ˆ ( ( w d a ( k σ y ( ( ( ( ˆ ( ( ( 3 w a abw g abw g w w w Now we can smmarze the algorthm. At any tme ont t, the otmal olcy and the vale of that olcy are: * ( t ( t ( t ( g ˆ ( t ( t ( t ( t ( t (, ˆ ( ˆ ( t ( t ( t 3 w w w At the last tme ont we have: ( At any other tme ont we have: ( ( ( α 3 g 0 w w 0 w 0 ( α ( ( ( ( ( ( ( ( ( α ( ( ( ( ( ( 3 σ y g w b c w b w b c w ab w w a w abg w w abg w w d a k
Hidden Markov Model Cheat Sheet
Hdden Markov Model Cheat Sheet (GIT ID: dc2f391536d67ed5847290d5250d4baae103487e) Ths document s a cheat sheet on Hdden Markov Models (HMMs). It resembles lecture notes, excet that t cuts to the chase
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationMixture of Gaussians Expectation Maximization (EM) Part 2
Mture of Gaussans Eectaton Mamaton EM Part 2 Most of the sldes are due to Chrstoher Bsho BCS Summer School Eeter 2003. The rest of the sldes are based on lecture notes by A. Ng Lmtatons of K-means Hard
More information( ) 2 ( ) ( ) Problem Set 4 Suggested Solutions. Problem 1
Problem Set 4 Suggested Solutons Problem (A) The market demand functon s the soluton to the followng utlty-maxmzaton roblem (UMP): The Lagrangean: ( x, x, x ) = + max U x, x, x x x x st.. x + x + x y x,
More informationManaging Capacity Through Reward Programs. on-line companion page. Byung-Do Kim Seoul National University College of Business Administration
Managng Caacty Through eward Programs on-lne comanon age Byung-Do Km Seoul Natonal Unversty College of Busness Admnstraton Mengze Sh Unversty of Toronto otman School of Management Toronto ON M5S E6 Canada
More informationThe Dirac Equation for a One-electron atom. In this section we will derive the Dirac equation for a one-electron atom.
The Drac Equaton for a One-electron atom In ths secton we wll derve the Drac equaton for a one-electron atom. Accordng to Ensten the energy of a artcle wth rest mass m movng wth a velocty V s gven by E
More informationTracking with Kalman Filter
Trackng wth Kalman Flter Scott T. Acton Vrgna Image and Vdeo Analyss (VIVA), Charles L. Brown Department of Electrcal and Computer Engneerng Department of Bomedcal Engneerng Unversty of Vrgna, Charlottesvlle,
More informationConservation of Angular Momentum = "Spin"
Page 1 of 6 Conservaton of Angular Momentum = "Spn" We can assgn a drecton to the angular velocty: drecton of = drecton of axs + rght hand rule (wth rght hand, curl fngers n drecton of rotaton, thumb ponts
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another well-known teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More informationTransfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system
Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More information6 Supplementary Materials
6 Supplementar Materals 61 Proof of Theorem 31 Proof Let m Xt z 1:T : l m Xt X,z 1:t Wethenhave mxt z1:t ˆm HX Xt z 1:T mxt z1:t m HX Xt z 1:T + mxt z 1:T HX We consder each of the two terms n equaton
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More information+, where 0 x N - n. k k
CO 745, Mdterm Len Cabrera. A multle choce eam has questons, each of whch has ossble answers. A student nows the correct answer to n of these questons. For the remanng - n questons, he checs the answers
More informationConfidence intervals for weighted polynomial calibrations
Confdence ntervals for weghted olynomal calbratons Sergey Maltsev, Amersand Ltd., Moscow, Russa; ur Kalambet, Amersand Internatonal, Inc., Beachwood, OH e-mal: kalambet@amersand-ntl.com htt://www.chromandsec.com
More informationAlgorithms for factoring
CSA E0 235: Crytograhy Arl 9,2015 Instructor: Arta Patra Algorthms for factorng Submtted by: Jay Oza, Nranjan Sngh Introducton Factorsaton of large ntegers has been a wdely studed toc manly because of
More informationComplex Numbers Practice 0708 & SP 1. The complex number z is defined by
IB Math Hgh Leel: Complex Nmbers Practce 0708 & SP Complex Nmbers Practce 0708 & SP. The complex nmber z s defned by π π π π z = sn sn. 6 6 Ale - Desert Academy (a) Express z n the form re, where r and
More information6. Hamilton s Equations
6. Hamlton s Equatons Mchael Fowler A Dynamcal System s Path n Confguraton Sace and n State Sace The story so far: For a mechancal system wth n degrees of freedom, the satal confguraton at some nstant
More informationGeometric Modeling
Geometrc Modelng 9.580.0 Notes on Crve and Srface Contnty Parts of Mortenson, Farn, Angel, Hll and others From Prevos Lectres Contnty at Jon Ponts (from Lectre ) Dscontnos: hyscal searaton Parametrc Contnty
More informationAE/ME 339. K. M. Isaac. 8/31/2004 topic4: Implicit method, Stability, ADI method. Computational Fluid Dynamics (AE/ME 339) MAEEM Dept.
AE/ME 339 Comptatonal Fld Dynamcs (CFD) Comptatonal Fld Dynamcs (AE/ME 339) Implct form of dfference eqaton In the prevos explct method, the solton at tme level n,,n, depended only on the known vales of,
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationFuzzy approach to solve multi-objective capacitated transportation problem
Internatonal Journal of Bonformatcs Research, ISSN: 0975 087, Volume, Issue, 00, -0-4 Fuzzy aroach to solve mult-objectve caactated transortaton roblem Lohgaonkar M. H. and Bajaj V. H.* * Deartment of
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationChapter Twelve. Integration. We now turn our attention to the idea of an integral in dimensions higher than one. Consider a real-valued function f : D
Chapter Twelve Integraton 12.1 Introducton We now turn our attenton to the dea of an ntegral n dmensons hgher than one. Consder a real-valued functon f : R, where the doman s a nce closed subset of Eucldean
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationPHYS 705: Classical Mechanics. Newtonian Mechanics
1 PHYS 705: Classcal Mechancs Newtonan Mechancs Quck Revew of Newtonan Mechancs Basc Descrpton: -An dealzed pont partcle or a system of pont partcles n an nertal reference frame [Rgd bodes (ch. 5 later)]
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationDynamic Systems on Graphs
Prepared by F.L. Lews Updated: Saturday, February 06, 200 Dynamc Systems on Graphs Control Graphs and Consensus A network s a set of nodes that collaborates to acheve what each cannot acheve alone. A network,
More informationCS 3750 Machine Learning Lecture 6. Monte Carlo methods. CS 3750 Advanced Machine Learning. Markov chain Monte Carlo
CS 3750 Machne Learnng Lectre 6 Monte Carlo methods Mlos Haskrecht mlos@cs.ptt.ed 5329 Sennott Sqare Markov chan Monte Carlo Importance samplng: samples are generated accordng to Q and every sample from
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationSo far: simple (planar) geometries
Physcs 06 ecture 5 Torque and Angular Momentum as Vectors SJ 7thEd.: Chap. to 3 Rotatonal quanttes as vectors Cross product Torque epressed as a vector Angular momentum defned Angular momentum as a vector
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationPattern Classification
attern Classfcaton All materals n these sldes were taken from attern Classfcaton nd ed by R. O. Duda,. E. Hart and D. G. Stork, John Wley & Sons, 000 wth the ermsson of the authors and the ublsher Chater
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More informationDigital PI Controller Equations
Ver. 4, 9 th March 7 Dgtal PI Controller Equatons Probably the most common tye of controller n ndustral ower electroncs s the PI (Proortonal - Integral) controller. In feld orented motor control, PI controllers
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationPattern Classification (II) 杜俊
attern lassfcaton II 杜俊 junu@ustc.eu.cn Revew roalty & Statstcs Bayes theorem Ranom varales: screte vs. contnuous roalty struton: DF an DF Statstcs: mean, varance, moment arameter estmaton: MLE Informaton
More informationMechanics Physics 151
Mechancs hyscs 151 Lecture Canoncal Transformatons (Chater 9) What We Dd Last Tme Drect Condtons Q j Q j = = j, Q, j, Q, Necessary and suffcent j j for Canoncal Transf. = = j Q, Q, j Q, Q, Infntesmal CT
More informationELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM
ELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM An elastc wave s a deformaton of the body that travels throughout the body n all drectons. We can examne the deformaton over a perod of tme by fxng our look
More informationSociété de Calcul Mathématique SA
Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc
More informationCS286r Assign One. Answer Key
CS286r Assgn One Answer Key 1 Game theory 1.1 1.1.1 Let off-equlbrum strateges also be that people contnue to play n Nash equlbrum. Devatng from any Nash equlbrum s a weakly domnated strategy. That s,
More informationAdvanced Topics in Optimization. Piecewise Linear Approximation of a Nonlinear Function
Advanced Tocs n Otmzaton Pecewse Lnear Aroxmaton of a Nonlnear Functon Otmzaton Methods: M8L Introducton and Objectves Introducton There exsts no general algorthm for nonlnear rogrammng due to ts rregular
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationLecture 14: Bandits with Budget Constraints
IEOR 8100-001: Learnng and Optmzaton for Sequental Decson Makng 03/07/16 Lecture 14: andts wth udget Constrants Instructor: Shpra Agrawal Scrbed by: Zhpeng Lu 1 Problem defnton In the regular Mult-armed
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationJAB Chain. Long-tail claims development. ASTIN - September 2005 B.Verdier A. Klinger
JAB Chan Long-tal clams development ASTIN - September 2005 B.Verder A. Klnger Outlne Chan Ladder : comments A frst soluton: Munch Chan Ladder JAB Chan Chan Ladder: Comments Black lne: average pad to ncurred
More informationResource Allocation and Decision Analysis (ECON 8010) Spring 2014 Foundations of Regression Analysis
Resource Allocaton and Decson Analss (ECON 800) Sprng 04 Foundatons of Regresson Analss Readng: Regresson Analss (ECON 800 Coursepak, Page 3) Defntons and Concepts: Regresson Analss statstcal technques
More informationPhysics 181. Particle Systems
Physcs 181 Partcle Systems Overvew In these notes we dscuss the varables approprate to the descrpton of systems of partcles, ther defntons, ther relatons, and ther conservatons laws. We consder a system
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More information( ) [ ( k) ( k) ( x) ( ) ( ) ( ) [ ] ξ [ ] [ ] [ ] ( )( ) i ( ) ( )( ) 2! ( ) = ( ) 3 Interpolation. Polynomial Approximation.
3 Interpolaton {( y } Gven:,,,,,, [ ] Fnd: y for some Mn, Ma Polynomal Appromaton Theorem (Weerstrass Appromaton Theorem --- estence ε [ ab] f( P( , then there ests a polynomal
More informationAGC Introduction
. Introducton AGC 3 The prmary controller response to a load/generaton mbalance results n generaton adjustment so as to mantan load/generaton balance. However, due to droop, t also results n a non-zero
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationχ x B E (c) Figure 2.1.1: (a) a material particle in a body, (b) a place in space, (c) a configuration of the body
Secton.. Moton.. The Materal Body and Moton hyscal materals n the real world are modeled usng an abstract mathematcal entty called a body. Ths body conssts of an nfnte number of materal partcles. Shown
More informationSupplementary Material for Spectral Clustering based on the graph p-laplacian
Sulementary Materal for Sectral Clusterng based on the grah -Lalacan Thomas Bühler and Matthas Hen Saarland Unversty, Saarbrücken, Germany {tb,hen}@csun-sbde May 009 Corrected verson, June 00 Abstract
More informationIterative General Dynamic Model for Serial-Link Manipulators
EEL6667: Knematcs, Dynamcs and Control of Robot Manpulators 1. Introducton Iteratve General Dynamc Model for Seral-Lnk Manpulators In ths set of notes, we are gong to develop a method for computng a general
More informationAdditional Codes using Finite Difference Method. 1 HJB Equation for Consumption-Saving Problem Without Uncertainty
Addtonal Codes usng Fnte Dfference Method Benamn Moll 1 HJB Equaton for Consumpton-Savng Problem Wthout Uncertanty Before consderng the case wth stochastc ncome n http://www.prnceton.edu/~moll/ HACTproect/HACT_Numercal_Appendx.pdf,
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationMidterm Examination. Regression and Forecasting Models
IOMS Department Regresson and Forecastng Models Professor Wllam Greene Phone: 22.998.0876 Offce: KMC 7-90 Home page: people.stern.nyu.edu/wgreene Emal: wgreene@stern.nyu.edu Course web page: people.stern.nyu.edu/wgreene/regresson/outlne.htm
More informationLecture 2: Prelude to the big shrink
Lecture 2: Prelude to the bg shrnk Last tme A slght detour wth vsualzaton tools (hey, t was the frst day... why not start out wth somethng pretty to look at?) Then, we consdered a smple 120a-style regresson
More informationCanonical transformations
Canoncal transformatons November 23, 2014 Recall that we have defned a symplectc transformaton to be any lnear transformaton M A B leavng the symplectc form nvarant, Ω AB M A CM B DΩ CD Coordnate transformatons,
More informationEPR Paradox and the Physical Meaning of an Experiment in Quantum Mechanics. Vesselin C. Noninski
EPR Paradox and the Physcal Meanng of an Experment n Quantum Mechancs Vesseln C Nonnsk vesselnnonnsk@verzonnet Abstract It s shown that there s one purely determnstc outcome when measurement s made on
More informationChapter 3 Describing Data Using Numerical Measures
Chapter 3 Student Lecture Notes 3-1 Chapter 3 Descrbng Data Usng Numercal Measures Fall 2006 Fundamentals of Busness Statstcs 1 Chapter Goals To establsh the usefulness of summary measures of data. The
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationThe Robustness of a Nash Equilibrium Simulation Model
8th World IMACS / MODSIM Congress, Carns, Australa 3-7 July 2009 htt://mssanz.org.au/modsm09 The Robustness of a Nash Equlbrum Smulaton Model Etaro Ayosh, Atsush Mak 2 and Takash Okamoto 3 Faculty of Scence
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle-
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationSTAT 511 FINAL EXAM NAME Spring 2001
STAT 5 FINAL EXAM NAME Sprng Instructons: Ths s a closed book exam. No notes or books are allowed. ou may use a calculator but you are not allowed to store notes or formulas n the calculator. Please wrte
More informationLecture 7: Boltzmann distribution & Thermodynamics of mixing
Prof. Tbbtt Lecture 7 etworks & Gels Lecture 7: Boltzmann dstrbuton & Thermodynamcs of mxng 1 Suggested readng Prof. Mark W. Tbbtt ETH Zürch 13 März 018 Molecular Drvng Forces Dll and Bromberg: Chapters
More informationHidden Markov Models
Hdden Markov Models Namrata Vaswan, Iowa State Unversty Aprl 24, 204 Hdden Markov Model Defntons and Examples Defntons:. A hdden Markov model (HMM) refers to a set of hdden states X 0, X,..., X t,...,
More informationThe Second Anti-Mathima on Game Theory
The Second Ant-Mathma on Game Theory Ath. Kehagas December 1 2006 1 Introducton In ths note we wll examne the noton of game equlbrum for three types of games 1. 2-player 2-acton zero-sum games 2. 2-player
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More informationSE Story Shear Frame. Final Project. 2 Story Bending Beam. m 2. u 2. m 1. u 1. m 3. u 3 L 3. Given: L 1 L 2. EI ω 1 ω 2 Solve for m 2.
SE 8 Fnal Project Story Sear Frame Gven: EI ω ω Solve for Story Bendng Beam Gven: EI ω ω 3 Story Sear Frame Gven: L 3 EI ω ω ω 3 3 m 3 L 3 Solve for Solve for m 3 3 4 3 Story Bendng Beam Part : Determnng
More informationQuantum Mechanics for Scientists and Engineers. David Miller
Quantum Mechancs for Scentsts and Engneers Davd Mller Types of lnear operators Types of lnear operators Blnear expanson of operators Blnear expanson of lnear operators We know that we can expand functons
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationDO NOT DO HOMEWORK UNTIL IT IS ASSIGNED. THE ASSIGNMENTS MAY CHANGE UNTIL ANNOUNCED.
EE 539 Homeworks Sprng 08 Updated: Tuesday, Aprl 7, 08 DO NOT DO HOMEWORK UNTIL IT IS ASSIGNED. THE ASSIGNMENTS MAY CHANGE UNTIL ANNOUNCED. For full credt, show all work. Some problems requre hand calculatons.
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationWhat Independencies does a Bayes Net Model? Bayesian Networks: Independencies and Inference. Quick proof that independence is symmetric
Bayesan Networks: Indeendences and Inference Scott Daves and ndrew Moore Note to other teachers and users of these sldes. ndrew and Scott would be delghted f you found ths source materal useful n gvng
More informationTopic 5: Non-Linear Regression
Topc 5: Non-Lnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually non-lnear n the parameters.
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More information5 The Rational Canonical Form
5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More information