Ω ). Then the following inequality takes place:

Size: px
Start display at page:

Download "Ω ). Then the following inequality takes place:"

Transcription

1 Lecture 8 Lemma 5. Let f : R R be a cotiuously differetiable covex fuctio. Choose a costat δ > ad cosider the subset Ωδ = { R f δ } R. Let Ωδ ad assume that f < δ, i.e., is ot o the boudary of f = δ, i.e., is o the boudary of δ Ω δ. Also, let Ωδ ad assume that Ω. he the followig iequality takes place: f 5.3 where f f = f R is the gradiet vector of f evaluated at. he iequality 5.3 is illustrated i Figure 5.. It shows that the gradiet vector evaluated at the boudary of a covex set always poits away from the set. f Ω δ Proof: Sice f is covex the f Figure 5.: Fuctio Gradiet λ + λ λ f + λ f or equivaletly: f + λ f + λ f f he for ay ozero < λ : 83

2 f + λ f = λ < δ δ f < δ δ = f akig the limit as λ yields the iequality 5.3 ad completes the proof. Let f be a covex radially ubouded fuctio. he oe ca show that for ay δ >, the set Ω = R f R is covex ad compact. { } δ δ We may ow itroduce the Proectio Operator for two vectors. or equivaletly: y, if f Pro, y = y, if f y f f f y y f, if ot. f 5.4 y Pro, f f f y y f, if f > y f > = y, if ot 5.5 Equivalece betwee 5.4 ad 5.5 is proved below: { f f y f } = f > f y f { f f y f } { }.NO..NO. = > < > = > < > > = > > f f f y f { f y f } Formal defiitio of the Proectio Operator for two matrices is stated below. Defiitio 5.3 he Proectio Operator N Let f : R R be a covex radially ubouded fuctio. Give two matrices N Θ= [ ] R ad [ ] N matrix, N Y = y y R, the Proectio Operator is a 84

3 Y y N yn Pro Θ, = Pro, Pro, 5.6 where y Pro, f f f y y f,if f y f > > = y,if f y f 5.7 represets the th colum of the operator. Remark 5. Geometrical iterpretatio of 5.5 ca be give as follows. Suppose that, the true parameter vector, belogs to the covex set Ω { f } Ω = R 5.8 Itroduce aother covex set: { f } Ω = R 5.9 It is obvious that Ω Ω. Defiitio 5.5 implies that the Proectio Operator Pro, y does ot alter the vector y if belogs to the covex set Ω i 5.8. O the Ω \ Ω = : f, the Proectio Operator other had, i the aulus set { } subtracts a vector ormal to the boudary { = λ} f from y so that oe gets a smooth trasformatio from the origial vector field y for λ = to a taget to the boudary vector field for λ =. his is show i Figure 5.3. { f = λ} Pro, y y f { f } 85

4 Figure 5.3: Proectio Operator Usig Lemma 5. ad the iequality 5.3, yields the followig importat property of the Proectio Operator: y Pro, y, if f =, if f ad y f f f y f f = λ, if ot. 5. or, equivaletly Pro, y his iequality ca be geeralized for matrices. Lemma 5.3 Let f be a covex fuctio. Let N Y = [ y y ] R y 5., N ΘΘ R be two matrices. he for ay matrix, the followig iequality takes place: { Y Y } trace Θ Θ Pro Θ, 5. Proof: Usig 5., oe immediately gets: m { } = trace Θ Θ Pro Θ, Y Y = Pro, Y Y 5.3 ad the proof is complete. Lemma 5.4 Let f be a covex radially ubouded fuctio. For a time-varyig piecewise cotiuous vector y t, cosider the followig IVP: 86

5 Pro, y R f = = Ω = { } 5.4 where Pro, y is the Proectio Operator defied as i 5.5: y Pro, he t R f Proof: f f f y y f, if f > y f > = y, if ot { } Ω =, for all t. It is sufficiet to show that f f t compute time derivative of Substitutig 5.5 ito 5.5, results i: f alog the traectories of 5.4: = = Pro,, for all t. owards this ed, f f f y 5.5 Pro, = f f y f y f f y f f y f y f, if > > =, if 5.6 Cosequetly: f >, if < f < y f > f =, if f = 5.7 he st ad the d relatios i 5.7 imply that if for Ω\ Ω f mootoically icreases, the fuctio value will ever exceed. I other words, if f the f t, for all t. his completes the proof of the Lemma. 87

6 Remark 5. he vector y t i 5.4 ca be viewed as the commaded velocity of the system state t. he Proectio Operator i 5.4 modifies the commaded velocity y oly i the aulus regio Ω\ Ω, such that t will ever leave Ω, for all future times. his is the mai beefit of the Proectio Operator. Remark 5.3 Suppose that <, where is some positive costat. If i the defiitio of the Proectio Operator 5.5, the covex radially ubouded fuctio f is chose as: where ε >, the f f = ad ε = 5.8 ε { R f } { R : } { R f } { R : ε} Ω = = Ω = = + I this case, Lemma 5.4 guaratees that t + ε future times. 5.9 for all 6. System ID usig Dyamic Model ad he Proectio Operator I this sectio, usig the Proectio Operator 5.7, we itroduce modified parameter estimatio laws such that the estimated parameters remai uiformly bouded i the presece of o-parametric ucertaities. Cosider the system dyamics i the form:,, x = f x u + F x u Kow Ukow 6. Assume that both the system state bouded i time, x R ad the cotrol iput m, : u m R are uiformly X U = x R u R x x u u 6. 88

7 where x, u are kow positive costats. For all, fuctio F xu, ca be writte as: x u X U, the ukow, =Θ Φ, + ε, F x u x u x u Ucertaity Parametric No-Parametric 6.3 where N Θ R N is the matrix of ukow costat parameters, Φ x, u R is the kow regressor vector, ad x, u ε R represets the fuctio approximatio error. If compoets of the regressor vector form satisfy the Uiversal Approximatio Property the give tolerace ε >, there exists a sufficietly large N ad a matrix of costat parameters N Θ R, such that the ukow fuctio, withi the tolerace ε,,, ε, ε for all pairs x, u from the compact set X system dyamics becomes: F xu ca be approximated F x u Θ Φ x u = x u 6.4 U defied i 6.. Usig 6.3, the he system state predictor dyamics is defied as: x = f x, u +Θ Φ x, u +ε x, u 6.5 x = A x x + f x, u +Θ Φ x, u 6.6 ref where A ref is a Hurwitz matrix, ad N Θ R is the matrix of estimated parameters. Subtractig 6.5 from 6.6, oe ca show that the dyamics of state predictio error ca be writte as: et = xt xt 6.7 where, ε, ref, ε, e = Aref e+ Θ Θ Φ x u + x u = A e+δθ Φ x u + x u ΔΘ 6.8 ΔΘ=Θ Θ 6.9 represets the parameter estimatio error. Robust parameter estimatio / adaptatio laws are formulated ext. 89

8 heorem 6. For every, let f true ukow matrix of parameters Θ= [,, ], as well as its iitial guess defied such that, Ω = R f, for all be a covex radially ubouded fuctio. Suppose that the { } Θ are. Choose a Hurwitz matrix A ref, a positive defiite symmetric matrix Q, ad compute the uique solutio P= P > of the algebraic Lyapuov equatio. PA + A P= Q 6. ref ref Cosider the followig parameter estimatio law: Θ Φ ep Θ=Γ Pro, Θ = 6. he for all ad for all t Moreover, e, Proof: Θ ad L { } t R f Ω = 6., t : x t x t P ε > 6.3 λmi Q he relatio 6. directly follows from Lemma 5.4. Cosider the Lyapuov fuctio cadidate:, e Pe trace V e ΔΘ = + ΔΘ Γ ΔΘ 6.4 where Γ=Γ > is the adaptatio rate ad e is the system state predictio error, whose dyamics are defied as:,, = +ΔΘ Φ +ε 6.5 e Aref e x u x u Computig the time derivative of V alog the traectories of 6. ad 6.5, yields: 9

9 ref ref ε ε V e, ΔΘ = A e+ ΔΘ Φ + x, u Pe+ e P A e+ ΔΘ Φ + x, u e PAref Aref e e e Pε x u + trace ΔΘ Γ Θ = + +, Q e P + e PΔΘ Φ+ trace ΔΘ Pro Θ, Φ e P ε trace Pro = e Qe+ ΔΘ Φ e P+ Θ, Φ e P + e Pε x, u Y Y 6.6 Usig the iequality 5. from Lemma 5.3, gives: Defie, λmi Q e V e, ΔΘ = e Qe + trace ΔΘ Pro Θ, Y Y + e P λ Q e + e P ε = e λ Q e P ε mi mi P ε E = e R : e λmi Q ε Due to 6., the time-varyig matrix of estimated parameters Θ t is uiformly ultimately bouded UUB. Cosequetly, ΔΘ L. Also, V e, ΔΘ < outside of E. hus, the predictio error et eters the compact set E i fiite time, ad will remai iside the set for all future times. I other words, e is UUB. his completes the proof of the theorem. 7. System ID Modificatios for Robustess Cosider the geeric parameter estimatio law Θ = ΓΦe 7. where Γ is the learig rate matrix, Φ is the regressor vector, ad e is the so-called traiig error sigal. For example, usig a static liear i parameters model, e was take as the model predictio error, while i the case of a dyamic model, the traiig error was represeted by e P. I Sectio 3, oparametric ucertaities were itroduced ito the estimatio problem. It was show that due to the presece of the oparametric ucertaities i the model, boudedess of the estimated parameters was i questio. 9

10 his the problem was addressed i the previous sectio, where the Proectio Operator was itroduced ad was prove to eforce uiform boudedess of the estimated parameters. Various other optios for robustifyig the parameter estimatio laws exist. I this sectio, 3 additioal modificatios are cosidered. hey are: σ Modificatio e Modificatio Dead-Zoe Modificatio I the σ modificatio, the estimatio law 7. is modified to: Θ= ΓΦ e Γσ Θ Θ 7. where σ > is a small positive costat ad Θ is a costat matrix which is ofte selected to be zero. Basically, the d term i 7. prevets Θ from driftig to. heoretically speakig, the estimatio laws with the σ mod provide uiform boudedess of the traiig error sigal ad of the estimated parameters i the presece of the oparametric ucertaities. However, the modificatio has udesirable effects: a it slows dow the estimatio process ad b whe the traiig error sigal becomes small the estimated parameters go back to Θ, that is the parameters forget what they have ust leared. he e modificatio was motivated to elimiate the drawbacks associated with the σ mod. It is give by: Θ= ΓΦe Γ e γ Θ Θ 7.3 where γ > ad Θ are the desig tuig kobs. he mai idea here is to zero out the dampig term whe the traiig error sigal becomes zero. If the estimated parameters start driftig to large values the, similar to the σ mod,, the e mod adds dampig ad eforces uiform boudedess of the estimated parameters, as well as of the traiig error. However, the e mod variable dampig sigificatly slows dow the estimatio process whe the traiig error is large. he mai idea behid the dead-zoe modificatio is to ehace robustess by turig off the estimatio process whe the traiig error becomes relatively small. he estimatio laws with the dead-zoe mod are give by:, if ΓΦ e e e Θ=, if e < e mi mi 7.4 9

11 where e mi is a positive desig costat which defies the desired upper boud o the traiig error. It is importat to ote that, that the σ mod, the e mod, ad the Proectio Operator do ot require ay assumptios about upper bouds o the model error, yet these methods do prevet the parameter estimates from divergig to ifiity. However, whe the traiig error is small relative to the model error, the three modificatios do ot guaratee to maitai the accuracy of the parameter estimates. O the other had, whe a boud o the model errors is kow, the dead-zoe modificatio maitais the parameter estimatio accuracy. For these reasos, ofte i practice the dead-zoe is combied with the other three robust modificatios. 93

CALIFORNIA INSTITUTE OF TECHNOLOGY. Control and Dynamical Systems CDS 270. Eugene Lavretsky Spring 2007

CALIFORNIA INSTITUTE OF TECHNOLOGY. Control and Dynamical Systems CDS 270. Eugene Lavretsky Spring 2007 CALIFORNIA INSTITUTE OF TECHNOLOGY Cotrol ad Dyamical Systems CDS 7 Eugee Lavretsky Sprig 7 Lecture 1 1. Itroductio Readig material: [1]: Chapter 1, Sectios 1.1, 1. [1]: Chapter 3, Sectio 3.1 []: Chapter

More information

1 Approximating Integrals using Taylor Polynomials

1 Approximating Integrals using Taylor Polynomials Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................

More information

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4. 4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad

More information

Lyapunov Stability Analysis for Feedback Control Design

Lyapunov Stability Analysis for Feedback Control Design Copyright F.L. Lewis 008 All rights reserved Updated: uesday, November, 008 Lyapuov Stability Aalysis for Feedbac Cotrol Desig Lyapuov heorems Lyapuov Aalysis allows oe to aalyze the stability of cotiuous-time

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short

More information

Recurrence Relations

Recurrence Relations Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The

More information

Principle Of Superposition

Principle Of Superposition ecture 5: PREIMINRY CONCEP O RUCUR NYI Priciple Of uperpositio Mathematically, the priciple of superpositio is stated as ( a ) G( a ) G( ) G a a or for a liear structural system, the respose at a give

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

PAPER : IIT-JAM 2010

PAPER : IIT-JAM 2010 MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

1.3 Convergence Theorems of Fourier Series. k k k k. N N k 1. With this in mind, we state (without proof) the convergence of Fourier series.

1.3 Convergence Theorems of Fourier Series. k k k k. N N k 1. With this in mind, we state (without proof) the convergence of Fourier series. .3 Covergece Theorems of Fourier Series I this sectio, we preset the covergece of Fourier series. A ifiite sum is, by defiitio, a limit of partial sums, that is, a cos( kx) b si( kx) lim a cos( kx) b si(

More information

b i u x i U a i j u x i u x j

b i u x i U a i j u x i u x j M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here

More information

Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12

Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12 Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Differentiable Convex Functions

Differentiable Convex Functions Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for

More information

Sequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence

Sequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet

More information

Machine Learning for Data Science (CS 4786)

Machine Learning for Data Science (CS 4786) Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm

More information

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero? 2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a

More information

Regression with quadratic loss

Regression with quadratic loss Regressio with quadratic loss Maxim Ragisky October 13, 2015 Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X,Y, where, as before,

More information

10-701/ Machine Learning Mid-term Exam Solution

10-701/ Machine Learning Mid-term Exam Solution 0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it

More information

TERMWISE DERIVATIVES OF COMPLEX FUNCTIONS

TERMWISE DERIVATIVES OF COMPLEX FUNCTIONS TERMWISE DERIVATIVES OF COMPLEX FUNCTIONS This writeup proves a result that has as oe cosequece that ay complex power series ca be differetiated term-by-term withi its disk of covergece The result has

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio

More information

3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense,

3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense, 3. Z Trasform Referece: Etire Chapter 3 of text. Recall that the Fourier trasform (FT) of a DT sigal x [ ] is ω ( ) [ ] X e = j jω k = xe I order for the FT to exist i the fiite magitude sese, S = x [

More information

REGRESSION WITH QUADRATIC LOSS

REGRESSION WITH QUADRATIC LOSS REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d

More information

Introduction to Signals and Systems, Part V: Lecture Summary

Introduction to Signals and Systems, Part V: Lecture Summary EEL33: Discrete-Time Sigals ad Systems Itroductio to Sigals ad Systems, Part V: Lecture Summary Itroductio to Sigals ad Systems, Part V: Lecture Summary So far we have oly looked at examples of o-recursive

More information

Discrete-Time Systems, LTI Systems, and Discrete-Time Convolution

Discrete-Time Systems, LTI Systems, and Discrete-Time Convolution EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [

More information

Lesson 10: Limits and Continuity

Lesson 10: Limits and Continuity www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals

More information

Math 508 Exam 2 Jerry L. Kazdan December 9, :00 10:20

Math 508 Exam 2 Jerry L. Kazdan December 9, :00 10:20 Math 58 Eam 2 Jerry L. Kazda December 9, 24 9: :2 Directios This eam has three parts. Part A has 8 True/False questio (2 poits each so total 6 poits), Part B has 5 shorter problems (6 poits each, so 3

More information

Sequences. A Sequence is a list of numbers written in order.

Sequences. A Sequence is a list of numbers written in order. Sequeces A Sequece is a list of umbers writte i order. {a, a 2, a 3,... } The sequece may be ifiite. The th term of the sequece is the th umber o the list. O the list above a = st term, a 2 = 2 d term,

More information

Chapter 7 Isoperimetric problem

Chapter 7 Isoperimetric problem Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated

More information

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial. Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable

More information

Adaptive Control: CDS 270 I

Adaptive Control: CDS 270 I Lecture Adative Cotrol: CDS 7 I CALIFORNIA INSTITUTE OF TECHNOLOGY Eugee Lavretsky Srig Robust ad Adative Cotrol Worksho Adative Cotrol: Itroductio, Overview, ad Alicatios E. Lavretsky Motivatig Eamle:

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Analytic Continuation

Analytic Continuation Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

Machine Learning Theory (CS 6783)

Machine Learning Theory (CS 6783) Machie Learig Theory (CS 6783) Lecture 2 : Learig Frameworks, Examples Settig up learig problems. X : istace space or iput space Examples: Computer Visio: Raw M N image vectorized X = 0, 255 M N, SIFT

More information

September 2012 C1 Note. C1 Notes (Edexcel) Copyright - For AS, A2 notes and IGCSE / GCSE worksheets 1

September 2012 C1 Note. C1 Notes (Edexcel) Copyright   - For AS, A2 notes and IGCSE / GCSE worksheets 1 September 0 s (Edecel) Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

Ma 530 Infinite Series I

Ma 530 Infinite Series I Ma 50 Ifiite Series I Please ote that i additio to the material below this lecture icorporated material from the Visual Calculus web site. The material o sequeces is at Visual Sequeces. (To use this li

More information

Problem Cosider the curve give parametrically as x = si t ad y = + cos t for» t» ß: (a) Describe the path this traverses: Where does it start (whe t =

Problem Cosider the curve give parametrically as x = si t ad y = + cos t for» t» ß: (a) Describe the path this traverses: Where does it start (whe t = Mathematics Summer Wilso Fial Exam August 8, ANSWERS Problem 1 (a) Fid the solutio to y +x y = e x x that satisfies y() = 5 : This is already i the form we used for a first order liear differetial equatio,

More information

Chapter 1. Complex Numbers. Dr. Pulak Sahoo

Chapter 1. Complex Numbers. Dr. Pulak Sahoo Chapter 1 Complex Numbers BY Dr. Pulak Sahoo Assistat Professor Departmet of Mathematics Uiversity Of Kalyai West Begal, Idia E-mail : sahoopulak1@gmail.com 1 Module-2: Stereographic Projectio 1 Euler

More information

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients. Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,

More information

Advanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology

Advanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4

More information

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3 MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special

More information

lim za n n = z lim a n n.

lim za n n = z lim a n n. Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget

More information

Math 113 Exam 3 Practice

Math 113 Exam 3 Practice Math Exam Practice Exam 4 will cover.-., 0. ad 0.. Note that eve though. was tested i exam, questios from that sectios may also be o this exam. For practice problems o., refer to the last review. This

More information

MA131 - Analysis 1. Workbook 3 Sequences II

MA131 - Analysis 1. Workbook 3 Sequences II MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................

More information

Quadratic Functions. Before we start looking at polynomials, we should know some common terminology.

Quadratic Functions. Before we start looking at polynomials, we should know some common terminology. Quadratic Fuctios I this sectio we begi the study of fuctios defied by polyomial expressios. Polyomial ad ratioal fuctios are the most commo fuctios used to model data, ad are used extesively i mathematical

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by

More information

Fall 2013 MTH431/531 Real analysis Section Notes

Fall 2013 MTH431/531 Real analysis Section Notes Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters

More information

Agnostic Learning and Concentration Inequalities

Agnostic Learning and Concentration Inequalities ECE901 Sprig 2004 Statistical Regularizatio ad Learig Theory Lecture: 7 Agostic Learig ad Cocetratio Iequalities Lecturer: Rob Nowak Scribe: Aravid Kailas 1 Itroductio 1.1 Motivatio I the last lecture

More information

Linear Support Vector Machines

Linear Support Vector Machines Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate

More information

Optimization Methods MIT 2.098/6.255/ Final exam

Optimization Methods MIT 2.098/6.255/ Final exam Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short

More information

Recursive Algorithms. Recurrences. Recursive Algorithms Analysis

Recursive Algorithms. Recurrences. Recursive Algorithms Analysis Recursive Algorithms Recurreces Computer Sciece & Egieerig 35: Discrete Mathematics Christopher M Bourke cbourke@cseuledu A recursive algorithm is oe i which objects are defied i terms of other objects

More information

The Growth of Functions. Theoretical Supplement

The Growth of Functions. Theoretical Supplement The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that

More information

Zeros of Polynomials

Zeros of Polynomials Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree

More information

Support vector machine revisited

Support vector machine revisited 6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector

More information

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) = AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,

More information

ECE 308 Discrete-Time Signals and Systems

ECE 308 Discrete-Time Signals and Systems ECE 38-5 ECE 38 Discrete-Time Sigals ad Systems Z. Aliyazicioglu Electrical ad Computer Egieerig Departmet Cal Poly Pomoa ECE 38-5 1 Additio, Multiplicatio, ad Scalig of Sequeces Amplitude Scalig: (A Costat

More information

( ) 1 Comparison Functions. α is strictly increasing since ( r) ( r ) α = for any positive real number c. = 0. It is said to belong to

( ) 1 Comparison Functions. α is strictly increasing since ( r) ( r ) α = for any positive real number c. = 0. It is said to belong to Compaiso Fuctios I this lesso, we study stability popeties of the oautoomous system = f t, x The difficulty is that ay solutio of this system statig at x( t ) depeds o both t ad t = x Thee ae thee special

More information

Algebra of Least Squares

Algebra of Least Squares October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal

More information

MIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS

MIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS MIDTERM 3 CALCULUS MATH 300 FALL 08 Moday, December 3, 08 5:5 PM to 6:45 PM Name PRACTICE EXAM S Please aswer all of the questios, ad show your work. You must explai your aswers to get credit. You will

More information

Most text will write ordinary derivatives using either Leibniz notation 2 3. y + 5y= e and y y. xx tt t

Most text will write ordinary derivatives using either Leibniz notation 2 3. y + 5y= e and y y. xx tt t Itroductio to Differetial Equatios Defiitios ad Termiolog Differetial Equatio: A equatio cotaiig the derivatives of oe or more depedet variables, with respect to oe or more idepedet variables, is said

More information

Solutions to Homework 7

Solutions to Homework 7 Solutios to Homework 7 Due Wedesday, August 4, 004. Chapter 4.1) 3, 4, 9, 0, 7, 30. Chapter 4.) 4, 9, 10, 11, 1. Chapter 4.1. Solutio to problem 3. The sum has the form a 1 a + a 3 with a k = 1/k. Sice

More information

( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let.

( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let. Joh Riley 8 Jue 03 ANSWERS TO EXERCISES IN APPENDIX B Sectio B VECTORS AND SETS Exercise B-: Covex sets (a) Let 0 x, x X, X, hece 0 x, x X ad 0 x, x X Sice X ad X are covex, x X ad x X The x X X, which

More information

MATHEMATICS. 61. The differential equation representing the family of curves where c is a positive parameter, is of

MATHEMATICS. 61. The differential equation representing the family of curves where c is a positive parameter, is of MATHEMATICS 6 The differetial equatio represetig the family of curves where c is a positive parameter, is of Order Order Degree (d) Degree (a,c) Give curve is y c ( c) Differetiate wrt, y c c y Hece differetial

More information

A widely used display of protein shapes is based on the coordinates of the alpha carbons - - C α

A widely used display of protein shapes is based on the coordinates of the alpha carbons - - C α Nice plottig of proteis: I A widely used display of protei shapes is based o the coordiates of the alpha carbos - - C α -s. The coordiates of the C α -s are coected by a cotiuous curve that roughly follows

More information

An Analysis of a Certain Linear First Order. Partial Differential Equation + f ( x, by Means of Topology

An Analysis of a Certain Linear First Order. Partial Differential Equation + f ( x, by Means of Topology Iteratioal Mathematical Forum 2 2007 o. 66 3241-3267 A Aalysis of a Certai Liear First Order Partial Differetial Equatio + f ( x y) = 0 z x by Meas of Topology z y T. Oepomo Sciece Egieerig ad Mathematics

More information

5. Likelihood Ratio Tests

5. Likelihood Ratio Tests 1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,

More information

CHAPTER 5. Theory and Solution Using Matrix Techniques

CHAPTER 5. Theory and Solution Using Matrix Techniques A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL

More information

6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.

6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer. 6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

Intro to Learning Theory

Intro to Learning Theory Lecture 1, October 18, 2016 Itro to Learig Theory Ruth Urer 1 Machie Learig ad Learig Theory Comig soo 2 Formal Framework 21 Basic otios I our formal model for machie learig, the istaces to be classified

More information

Ma 530 Introduction to Power Series

Ma 530 Introduction to Power Series Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Patter Recogitio Classificatio: No-Parametric Modelig Hamid R. Rabiee Jafar Muhammadi Sprig 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Ageda Parametric Modelig No-Parametric Modelig

More information

INEQUALITIES BJORN POONEN

INEQUALITIES BJORN POONEN INEQUALITIES BJORN POONEN 1 The AM-GM iequality The most basic arithmetic mea-geometric mea (AM-GM) iequality states simply that if x ad y are oegative real umbers, the (x + y)/2 xy, with equality if ad

More information

A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence

A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as

More information

Vector Quantization: a Limiting Case of EM

Vector Quantization: a Limiting Case of EM . Itroductio & defiitios Assume that you are give a data set X = { x j }, j { 2,,, }, of d -dimesioal vectors. The vector quatizatio (VQ) problem requires that we fid a set of prototype vectors Z = { z

More information

The picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled

The picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled 1 Lecture : Area Area ad distace traveled Approximatig area by rectagles Summatio The area uder a parabola 1.1 Area ad distace Suppose we have the followig iformatio about the velocity of a particle, how

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + 62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of

More information

Lecture 8: Solving the Heat, Laplace and Wave equations using finite difference methods

Lecture 8: Solving the Heat, Laplace and Wave equations using finite difference methods Itroductory lecture otes o Partial Differetial Equatios - c Athoy Peirce. Not to be copied, used, or revised without explicit writte permissio from the copyright ower. 1 Lecture 8: Solvig the Heat, Laplace

More information

THE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.

THE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0. THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of

More information

Statistics 511 Additional Materials

Statistics 511 Additional Materials Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

MATH 1080: Calculus of One Variable II Fall 2017 Textbook: Single Variable Calculus: Early Transcendentals, 7e, by James Stewart.

MATH 1080: Calculus of One Variable II Fall 2017 Textbook: Single Variable Calculus: Early Transcendentals, 7e, by James Stewart. MATH 1080: Calculus of Oe Variable II Fall 2017 Textbook: Sigle Variable Calculus: Early Trascedetals, 7e, by James Stewart Uit 3 Skill Set Importat: Studets should expect test questios that require a

More information

Math 312 Lecture Notes One Dimensional Maps

Math 312 Lecture Notes One Dimensional Maps Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,

More information

ENGI Series Page 6-01

ENGI Series Page 6-01 ENGI 3425 6 Series Page 6-01 6. Series Cotets: 6.01 Sequeces; geeral term, limits, covergece 6.02 Series; summatio otatio, covergece, divergece test 6.03 Stadard Series; telescopig series, geometric series,

More information

Introduction to Machine Learning DIS10

Introduction to Machine Learning DIS10 CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig

More information

Maximum Likelihood Estimation and Complexity Regularization

Maximum Likelihood Estimation and Complexity Regularization ECE90 Sprig 004 Statistical Regularizatio ad Learig Theory Lecture: 4 Maximum Likelihood Estimatio ad Complexity Regularizatio Lecturer: Rob Nowak Scribe: Pam Limpiti Review : Maximum Likelihood Estimatio

More information

Poincaré Problem for Nonlinear Elliptic Equations of Second Order in Unbounded Domains

Poincaré Problem for Nonlinear Elliptic Equations of Second Order in Unbounded Domains Advaces i Pure Mathematics 23 3 72-77 http://dxdoiorg/4236/apm233a24 Published Olie Jauary 23 (http://wwwscirporg/oural/apm) Poicaré Problem for Noliear Elliptic Equatios of Secod Order i Ubouded Domais

More information

Power Series: A power series about the center, x = 0, is a function of x of the form

Power Series: A power series about the center, x = 0, is a function of x of the form You are familiar with polyomial fuctios, polyomial that has ifiitely may terms. 2 p ( ) a0 a a 2 a. A power series is just a Power Series: A power series about the ceter, = 0, is a fuctio of of the form

More information

Frequency Response of FIR Filters

Frequency Response of FIR Filters EEL335: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we itroduce the idea of the frequecy respose of LTI systems, ad focus specifically o the frequecy respose of FIR filters.. Steady-state

More information

S1 Notation and Assumptions

S1 Notation and Assumptions Statistica Siica: Supplemet Robust-BD Estimatio ad Iferece for Varyig-Dimesioal Geeral Liear Models Chumig Zhag Xiao Guo Che Cheg Zhegju Zhag Uiversity of Wiscosi-Madiso Supplemetary Material S Notatio

More information

1 Hash tables. 1.1 Implementation

1 Hash tables. 1.1 Implementation Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

A Simplified Binet Formula for k-generalized Fibonacci Numbers

A Simplified Binet Formula for k-generalized Fibonacci Numbers A Simplified Biet Formula for k-geeralized Fiboacci Numbers Gregory P. B. Dresde Departmet of Mathematics Washigto ad Lee Uiversity Lexigto, VA 440 dresdeg@wlu.edu Zhaohui Du Shaghai, Chia zhao.hui.du@gmail.com

More information

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where

More information