Ω ). Then the following inequality takes place:
|
|
- Shanon Kennedy
- 5 years ago
- Views:
Transcription
1 Lecture 8 Lemma 5. Let f : R R be a cotiuously differetiable covex fuctio. Choose a costat δ > ad cosider the subset Ωδ = { R f δ } R. Let Ωδ ad assume that f < δ, i.e., is ot o the boudary of f = δ, i.e., is o the boudary of δ Ω δ. Also, let Ωδ ad assume that Ω. he the followig iequality takes place: f 5.3 where f f = f R is the gradiet vector of f evaluated at. he iequality 5.3 is illustrated i Figure 5.. It shows that the gradiet vector evaluated at the boudary of a covex set always poits away from the set. f Ω δ Proof: Sice f is covex the f Figure 5.: Fuctio Gradiet λ + λ λ f + λ f or equivaletly: f + λ f + λ f f he for ay ozero < λ : 83
2 f + λ f = λ < δ δ f < δ δ = f akig the limit as λ yields the iequality 5.3 ad completes the proof. Let f be a covex radially ubouded fuctio. he oe ca show that for ay δ >, the set Ω = R f R is covex ad compact. { } δ δ We may ow itroduce the Proectio Operator for two vectors. or equivaletly: y, if f Pro, y = y, if f y f f f y y f, if ot. f 5.4 y Pro, f f f y y f, if f > y f > = y, if ot 5.5 Equivalece betwee 5.4 ad 5.5 is proved below: { f f y f } = f > f y f { f f y f } { }.NO..NO. = > < > = > < > > = > > f f f y f { f y f } Formal defiitio of the Proectio Operator for two matrices is stated below. Defiitio 5.3 he Proectio Operator N Let f : R R be a covex radially ubouded fuctio. Give two matrices N Θ= [ ] R ad [ ] N matrix, N Y = y y R, the Proectio Operator is a 84
3 Y y N yn Pro Θ, = Pro, Pro, 5.6 where y Pro, f f f y y f,if f y f > > = y,if f y f 5.7 represets the th colum of the operator. Remark 5. Geometrical iterpretatio of 5.5 ca be give as follows. Suppose that, the true parameter vector, belogs to the covex set Ω { f } Ω = R 5.8 Itroduce aother covex set: { f } Ω = R 5.9 It is obvious that Ω Ω. Defiitio 5.5 implies that the Proectio Operator Pro, y does ot alter the vector y if belogs to the covex set Ω i 5.8. O the Ω \ Ω = : f, the Proectio Operator other had, i the aulus set { } subtracts a vector ormal to the boudary { = λ} f from y so that oe gets a smooth trasformatio from the origial vector field y for λ = to a taget to the boudary vector field for λ =. his is show i Figure 5.3. { f = λ} Pro, y y f { f } 85
4 Figure 5.3: Proectio Operator Usig Lemma 5. ad the iequality 5.3, yields the followig importat property of the Proectio Operator: y Pro, y, if f =, if f ad y f f f y f f = λ, if ot. 5. or, equivaletly Pro, y his iequality ca be geeralized for matrices. Lemma 5.3 Let f be a covex fuctio. Let N Y = [ y y ] R y 5., N ΘΘ R be two matrices. he for ay matrix, the followig iequality takes place: { Y Y } trace Θ Θ Pro Θ, 5. Proof: Usig 5., oe immediately gets: m { } = trace Θ Θ Pro Θ, Y Y = Pro, Y Y 5.3 ad the proof is complete. Lemma 5.4 Let f be a covex radially ubouded fuctio. For a time-varyig piecewise cotiuous vector y t, cosider the followig IVP: 86
5 Pro, y R f = = Ω = { } 5.4 where Pro, y is the Proectio Operator defied as i 5.5: y Pro, he t R f Proof: f f f y y f, if f > y f > = y, if ot { } Ω =, for all t. It is sufficiet to show that f f t compute time derivative of Substitutig 5.5 ito 5.5, results i: f alog the traectories of 5.4: = = Pro,, for all t. owards this ed, f f f y 5.5 Pro, = f f y f y f f y f f y f y f, if > > =, if 5.6 Cosequetly: f >, if < f < y f > f =, if f = 5.7 he st ad the d relatios i 5.7 imply that if for Ω\ Ω f mootoically icreases, the fuctio value will ever exceed. I other words, if f the f t, for all t. his completes the proof of the Lemma. 87
6 Remark 5. he vector y t i 5.4 ca be viewed as the commaded velocity of the system state t. he Proectio Operator i 5.4 modifies the commaded velocity y oly i the aulus regio Ω\ Ω, such that t will ever leave Ω, for all future times. his is the mai beefit of the Proectio Operator. Remark 5.3 Suppose that <, where is some positive costat. If i the defiitio of the Proectio Operator 5.5, the covex radially ubouded fuctio f is chose as: where ε >, the f f = ad ε = 5.8 ε { R f } { R : } { R f } { R : ε} Ω = = Ω = = + I this case, Lemma 5.4 guaratees that t + ε future times. 5.9 for all 6. System ID usig Dyamic Model ad he Proectio Operator I this sectio, usig the Proectio Operator 5.7, we itroduce modified parameter estimatio laws such that the estimated parameters remai uiformly bouded i the presece of o-parametric ucertaities. Cosider the system dyamics i the form:,, x = f x u + F x u Kow Ukow 6. Assume that both the system state bouded i time, x R ad the cotrol iput m, : u m R are uiformly X U = x R u R x x u u 6. 88
7 where x, u are kow positive costats. For all, fuctio F xu, ca be writte as: x u X U, the ukow, =Θ Φ, + ε, F x u x u x u Ucertaity Parametric No-Parametric 6.3 where N Θ R N is the matrix of ukow costat parameters, Φ x, u R is the kow regressor vector, ad x, u ε R represets the fuctio approximatio error. If compoets of the regressor vector form satisfy the Uiversal Approximatio Property the give tolerace ε >, there exists a sufficietly large N ad a matrix of costat parameters N Θ R, such that the ukow fuctio, withi the tolerace ε,,, ε, ε for all pairs x, u from the compact set X system dyamics becomes: F xu ca be approximated F x u Θ Φ x u = x u 6.4 U defied i 6.. Usig 6.3, the he system state predictor dyamics is defied as: x = f x, u +Θ Φ x, u +ε x, u 6.5 x = A x x + f x, u +Θ Φ x, u 6.6 ref where A ref is a Hurwitz matrix, ad N Θ R is the matrix of estimated parameters. Subtractig 6.5 from 6.6, oe ca show that the dyamics of state predictio error ca be writte as: et = xt xt 6.7 where, ε, ref, ε, e = Aref e+ Θ Θ Φ x u + x u = A e+δθ Φ x u + x u ΔΘ 6.8 ΔΘ=Θ Θ 6.9 represets the parameter estimatio error. Robust parameter estimatio / adaptatio laws are formulated ext. 89
8 heorem 6. For every, let f true ukow matrix of parameters Θ= [,, ], as well as its iitial guess defied such that, Ω = R f, for all be a covex radially ubouded fuctio. Suppose that the { } Θ are. Choose a Hurwitz matrix A ref, a positive defiite symmetric matrix Q, ad compute the uique solutio P= P > of the algebraic Lyapuov equatio. PA + A P= Q 6. ref ref Cosider the followig parameter estimatio law: Θ Φ ep Θ=Γ Pro, Θ = 6. he for all ad for all t Moreover, e, Proof: Θ ad L { } t R f Ω = 6., t : x t x t P ε > 6.3 λmi Q he relatio 6. directly follows from Lemma 5.4. Cosider the Lyapuov fuctio cadidate:, e Pe trace V e ΔΘ = + ΔΘ Γ ΔΘ 6.4 where Γ=Γ > is the adaptatio rate ad e is the system state predictio error, whose dyamics are defied as:,, = +ΔΘ Φ +ε 6.5 e Aref e x u x u Computig the time derivative of V alog the traectories of 6. ad 6.5, yields: 9
9 ref ref ε ε V e, ΔΘ = A e+ ΔΘ Φ + x, u Pe+ e P A e+ ΔΘ Φ + x, u e PAref Aref e e e Pε x u + trace ΔΘ Γ Θ = + +, Q e P + e PΔΘ Φ+ trace ΔΘ Pro Θ, Φ e P ε trace Pro = e Qe+ ΔΘ Φ e P+ Θ, Φ e P + e Pε x, u Y Y 6.6 Usig the iequality 5. from Lemma 5.3, gives: Defie, λmi Q e V e, ΔΘ = e Qe + trace ΔΘ Pro Θ, Y Y + e P λ Q e + e P ε = e λ Q e P ε mi mi P ε E = e R : e λmi Q ε Due to 6., the time-varyig matrix of estimated parameters Θ t is uiformly ultimately bouded UUB. Cosequetly, ΔΘ L. Also, V e, ΔΘ < outside of E. hus, the predictio error et eters the compact set E i fiite time, ad will remai iside the set for all future times. I other words, e is UUB. his completes the proof of the theorem. 7. System ID Modificatios for Robustess Cosider the geeric parameter estimatio law Θ = ΓΦe 7. where Γ is the learig rate matrix, Φ is the regressor vector, ad e is the so-called traiig error sigal. For example, usig a static liear i parameters model, e was take as the model predictio error, while i the case of a dyamic model, the traiig error was represeted by e P. I Sectio 3, oparametric ucertaities were itroduced ito the estimatio problem. It was show that due to the presece of the oparametric ucertaities i the model, boudedess of the estimated parameters was i questio. 9
10 his the problem was addressed i the previous sectio, where the Proectio Operator was itroduced ad was prove to eforce uiform boudedess of the estimated parameters. Various other optios for robustifyig the parameter estimatio laws exist. I this sectio, 3 additioal modificatios are cosidered. hey are: σ Modificatio e Modificatio Dead-Zoe Modificatio I the σ modificatio, the estimatio law 7. is modified to: Θ= ΓΦ e Γσ Θ Θ 7. where σ > is a small positive costat ad Θ is a costat matrix which is ofte selected to be zero. Basically, the d term i 7. prevets Θ from driftig to. heoretically speakig, the estimatio laws with the σ mod provide uiform boudedess of the traiig error sigal ad of the estimated parameters i the presece of the oparametric ucertaities. However, the modificatio has udesirable effects: a it slows dow the estimatio process ad b whe the traiig error sigal becomes small the estimated parameters go back to Θ, that is the parameters forget what they have ust leared. he e modificatio was motivated to elimiate the drawbacks associated with the σ mod. It is give by: Θ= ΓΦe Γ e γ Θ Θ 7.3 where γ > ad Θ are the desig tuig kobs. he mai idea here is to zero out the dampig term whe the traiig error sigal becomes zero. If the estimated parameters start driftig to large values the, similar to the σ mod,, the e mod adds dampig ad eforces uiform boudedess of the estimated parameters, as well as of the traiig error. However, the e mod variable dampig sigificatly slows dow the estimatio process whe the traiig error is large. he mai idea behid the dead-zoe modificatio is to ehace robustess by turig off the estimatio process whe the traiig error becomes relatively small. he estimatio laws with the dead-zoe mod are give by:, if ΓΦ e e e Θ=, if e < e mi mi 7.4 9
11 where e mi is a positive desig costat which defies the desired upper boud o the traiig error. It is importat to ote that, that the σ mod, the e mod, ad the Proectio Operator do ot require ay assumptios about upper bouds o the model error, yet these methods do prevet the parameter estimates from divergig to ifiity. However, whe the traiig error is small relative to the model error, the three modificatios do ot guaratee to maitai the accuracy of the parameter estimates. O the other had, whe a boud o the model errors is kow, the dead-zoe modificatio maitais the parameter estimatio accuracy. For these reasos, ofte i practice the dead-zoe is combied with the other three robust modificatios. 93
CALIFORNIA INSTITUTE OF TECHNOLOGY. Control and Dynamical Systems CDS 270. Eugene Lavretsky Spring 2007
CALIFORNIA INSTITUTE OF TECHNOLOGY Cotrol ad Dyamical Systems CDS 7 Eugee Lavretsky Sprig 7 Lecture 1 1. Itroductio Readig material: [1]: Chapter 1, Sectios 1.1, 1. [1]: Chapter 3, Sectio 3.1 []: Chapter
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationLyapunov Stability Analysis for Feedback Control Design
Copyright F.L. Lewis 008 All rights reserved Updated: uesday, November, 008 Lyapuov Stability Aalysis for Feedbac Cotrol Desig Lyapuov heorems Lyapuov Aalysis allows oe to aalyze the stability of cotiuous-time
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationSummary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector
Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short
More informationRecurrence Relations
Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The
More informationPrinciple Of Superposition
ecture 5: PREIMINRY CONCEP O RUCUR NYI Priciple Of uperpositio Mathematically, the priciple of superpositio is stated as ( a ) G( a ) G( ) G a a or for a liear structural system, the respose at a give
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More information1.3 Convergence Theorems of Fourier Series. k k k k. N N k 1. With this in mind, we state (without proof) the convergence of Fourier series.
.3 Covergece Theorems of Fourier Series I this sectio, we preset the covergece of Fourier series. A ifiite sum is, by defiitio, a limit of partial sums, that is, a cos( kx) b si( kx) lim a cos( kx) b si(
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationDifferentiable Convex Functions
Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationRegression with quadratic loss
Regressio with quadratic loss Maxim Ragisky October 13, 2015 Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X,Y, where, as before,
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationTERMWISE DERIVATIVES OF COMPLEX FUNCTIONS
TERMWISE DERIVATIVES OF COMPLEX FUNCTIONS This writeup proves a result that has as oe cosequece that ay complex power series ca be differetiated term-by-term withi its disk of covergece The result has
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More information3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense,
3. Z Trasform Referece: Etire Chapter 3 of text. Recall that the Fourier trasform (FT) of a DT sigal x [ ] is ω ( ) [ ] X e = j jω k = xe I order for the FT to exist i the fiite magitude sese, S = x [
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More informationIntroduction to Signals and Systems, Part V: Lecture Summary
EEL33: Discrete-Time Sigals ad Systems Itroductio to Sigals ad Systems, Part V: Lecture Summary Itroductio to Sigals ad Systems, Part V: Lecture Summary So far we have oly looked at examples of o-recursive
More informationDiscrete-Time Systems, LTI Systems, and Discrete-Time Convolution
EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [
More informationLesson 10: Limits and Continuity
www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals
More informationMath 508 Exam 2 Jerry L. Kazdan December 9, :00 10:20
Math 58 Eam 2 Jerry L. Kazda December 9, 24 9: :2 Directios This eam has three parts. Part A has 8 True/False questio (2 poits each so total 6 poits), Part B has 5 shorter problems (6 poits each, so 3
More informationSequences. A Sequence is a list of numbers written in order.
Sequeces A Sequece is a list of umbers writte i order. {a, a 2, a 3,... } The sequece may be ifiite. The th term of the sequece is the th umber o the list. O the list above a = st term, a 2 = 2 d term,
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationAdaptive Control: CDS 270 I
Lecture Adative Cotrol: CDS 7 I CALIFORNIA INSTITUTE OF TECHNOLOGY Eugee Lavretsky Srig Robust ad Adative Cotrol Worksho Adative Cotrol: Itroductio, Overview, ad Alicatios E. Lavretsky Motivatig Eamle:
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationAnalytic Continuation
Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationMachine Learning Theory (CS 6783)
Machie Learig Theory (CS 6783) Lecture 2 : Learig Frameworks, Examples Settig up learig problems. X : istace space or iput space Examples: Computer Visio: Raw M N image vectorized X = 0, 255 M N, SIFT
More informationSeptember 2012 C1 Note. C1 Notes (Edexcel) Copyright - For AS, A2 notes and IGCSE / GCSE worksheets 1
September 0 s (Edecel) Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationMa 530 Infinite Series I
Ma 50 Ifiite Series I Please ote that i additio to the material below this lecture icorporated material from the Visual Calculus web site. The material o sequeces is at Visual Sequeces. (To use this li
More informationProblem Cosider the curve give parametrically as x = si t ad y = + cos t for» t» ß: (a) Describe the path this traverses: Where does it start (whe t =
Mathematics Summer Wilso Fial Exam August 8, ANSWERS Problem 1 (a) Fid the solutio to y +x y = e x x that satisfies y() = 5 : This is already i the form we used for a first order liear differetial equatio,
More informationChapter 1. Complex Numbers. Dr. Pulak Sahoo
Chapter 1 Complex Numbers BY Dr. Pulak Sahoo Assistat Professor Departmet of Mathematics Uiversity Of Kalyai West Begal, Idia E-mail : sahoopulak1@gmail.com 1 Module-2: Stereographic Projectio 1 Euler
More informationDefinitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam 4 will cover.-., 0. ad 0.. Note that eve though. was tested i exam, questios from that sectios may also be o this exam. For practice problems o., refer to the last review. This
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationQuadratic Functions. Before we start looking at polynomials, we should know some common terminology.
Quadratic Fuctios I this sectio we begi the study of fuctios defied by polyomial expressios. Polyomial ad ratioal fuctios are the most commo fuctios used to model data, ad are used extesively i mathematical
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationAgnostic Learning and Concentration Inequalities
ECE901 Sprig 2004 Statistical Regularizatio ad Learig Theory Lecture: 7 Agostic Learig ad Cocetratio Iequalities Lecturer: Rob Nowak Scribe: Aravid Kailas 1 Itroductio 1.1 Motivatio I the last lecture
More informationLinear Support Vector Machines
Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate
More informationOptimization Methods MIT 2.098/6.255/ Final exam
Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short
More informationRecursive Algorithms. Recurrences. Recursive Algorithms Analysis
Recursive Algorithms Recurreces Computer Sciece & Egieerig 35: Discrete Mathematics Christopher M Bourke cbourke@cseuledu A recursive algorithm is oe i which objects are defied i terms of other objects
More informationThe Growth of Functions. Theoretical Supplement
The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that
More informationZeros of Polynomials
Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree
More informationSupport vector machine revisited
6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector
More informationNICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =
AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,
More informationECE 308 Discrete-Time Signals and Systems
ECE 38-5 ECE 38 Discrete-Time Sigals ad Systems Z. Aliyazicioglu Electrical ad Computer Egieerig Departmet Cal Poly Pomoa ECE 38-5 1 Additio, Multiplicatio, ad Scalig of Sequeces Amplitude Scalig: (A Costat
More information( ) 1 Comparison Functions. α is strictly increasing since ( r) ( r ) α = for any positive real number c. = 0. It is said to belong to
Compaiso Fuctios I this lesso, we study stability popeties of the oautoomous system = f t, x The difficulty is that ay solutio of this system statig at x( t ) depeds o both t ad t = x Thee ae thee special
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationMIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS
MIDTERM 3 CALCULUS MATH 300 FALL 08 Moday, December 3, 08 5:5 PM to 6:45 PM Name PRACTICE EXAM S Please aswer all of the questios, ad show your work. You must explai your aswers to get credit. You will
More informationMost text will write ordinary derivatives using either Leibniz notation 2 3. y + 5y= e and y y. xx tt t
Itroductio to Differetial Equatios Defiitios ad Termiolog Differetial Equatio: A equatio cotaiig the derivatives of oe or more depedet variables, with respect to oe or more idepedet variables, is said
More informationSolutions to Homework 7
Solutios to Homework 7 Due Wedesday, August 4, 004. Chapter 4.1) 3, 4, 9, 0, 7, 30. Chapter 4.) 4, 9, 10, 11, 1. Chapter 4.1. Solutio to problem 3. The sum has the form a 1 a + a 3 with a k = 1/k. Sice
More information( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let.
Joh Riley 8 Jue 03 ANSWERS TO EXERCISES IN APPENDIX B Sectio B VECTORS AND SETS Exercise B-: Covex sets (a) Let 0 x, x X, X, hece 0 x, x X ad 0 x, x X Sice X ad X are covex, x X ad x X The x X X, which
More informationMATHEMATICS. 61. The differential equation representing the family of curves where c is a positive parameter, is of
MATHEMATICS 6 The differetial equatio represetig the family of curves where c is a positive parameter, is of Order Order Degree (d) Degree (a,c) Give curve is y c ( c) Differetiate wrt, y c c y Hece differetial
More informationA widely used display of protein shapes is based on the coordinates of the alpha carbons - - C α
Nice plottig of proteis: I A widely used display of protei shapes is based o the coordiates of the alpha carbos - - C α -s. The coordiates of the C α -s are coected by a cotiuous curve that roughly follows
More informationAn Analysis of a Certain Linear First Order. Partial Differential Equation + f ( x, by Means of Topology
Iteratioal Mathematical Forum 2 2007 o. 66 3241-3267 A Aalysis of a Certai Liear First Order Partial Differetial Equatio + f ( x y) = 0 z x by Meas of Topology z y T. Oepomo Sciece Egieerig ad Mathematics
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationIntro to Learning Theory
Lecture 1, October 18, 2016 Itro to Learig Theory Ruth Urer 1 Machie Learig ad Learig Theory Comig soo 2 Formal Framework 21 Basic otios I our formal model for machie learig, the istaces to be classified
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationStatistical Pattern Recognition
Statistical Patter Recogitio Classificatio: No-Parametric Modelig Hamid R. Rabiee Jafar Muhammadi Sprig 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Ageda Parametric Modelig No-Parametric Modelig
More informationINEQUALITIES BJORN POONEN
INEQUALITIES BJORN POONEN 1 The AM-GM iequality The most basic arithmetic mea-geometric mea (AM-GM) iequality states simply that if x ad y are oegative real umbers, the (x + y)/2 xy, with equality if ad
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationVector Quantization: a Limiting Case of EM
. Itroductio & defiitios Assume that you are give a data set X = { x j }, j { 2,,, }, of d -dimesioal vectors. The vector quatizatio (VQ) problem requires that we fid a set of prototype vectors Z = { z
More informationThe picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled
1 Lecture : Area Area ad distace traveled Approximatig area by rectagles Summatio The area uder a parabola 1.1 Area ad distace Suppose we have the followig iformatio about the velocity of a particle, how
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationLecture 8: Solving the Heat, Laplace and Wave equations using finite difference methods
Itroductory lecture otes o Partial Differetial Equatios - c Athoy Peirce. Not to be copied, used, or revised without explicit writte permissio from the copyright ower. 1 Lecture 8: Solvig the Heat, Laplace
More informationTHE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.
THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationMATH 1080: Calculus of One Variable II Fall 2017 Textbook: Single Variable Calculus: Early Transcendentals, 7e, by James Stewart.
MATH 1080: Calculus of Oe Variable II Fall 2017 Textbook: Sigle Variable Calculus: Early Trascedetals, 7e, by James Stewart Uit 3 Skill Set Importat: Studets should expect test questios that require a
More informationMath 312 Lecture Notes One Dimensional Maps
Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,
More informationENGI Series Page 6-01
ENGI 3425 6 Series Page 6-01 6. Series Cotets: 6.01 Sequeces; geeral term, limits, covergece 6.02 Series; summatio otatio, covergece, divergece test 6.03 Stadard Series; telescopig series, geometric series,
More informationIntroduction to Machine Learning DIS10
CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig
More informationMaximum Likelihood Estimation and Complexity Regularization
ECE90 Sprig 004 Statistical Regularizatio ad Learig Theory Lecture: 4 Maximum Likelihood Estimatio ad Complexity Regularizatio Lecturer: Rob Nowak Scribe: Pam Limpiti Review : Maximum Likelihood Estimatio
More informationPoincaré Problem for Nonlinear Elliptic Equations of Second Order in Unbounded Domains
Advaces i Pure Mathematics 23 3 72-77 http://dxdoiorg/4236/apm233a24 Published Olie Jauary 23 (http://wwwscirporg/oural/apm) Poicaré Problem for Noliear Elliptic Equatios of Secod Order i Ubouded Domais
More informationPower Series: A power series about the center, x = 0, is a function of x of the form
You are familiar with polyomial fuctios, polyomial that has ifiitely may terms. 2 p ( ) a0 a a 2 a. A power series is just a Power Series: A power series about the ceter, = 0, is a fuctio of of the form
More informationFrequency Response of FIR Filters
EEL335: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we itroduce the idea of the frequecy respose of LTI systems, ad focus specifically o the frequecy respose of FIR filters.. Steady-state
More informationS1 Notation and Assumptions
Statistica Siica: Supplemet Robust-BD Estimatio ad Iferece for Varyig-Dimesioal Geeral Liear Models Chumig Zhag Xiao Guo Che Cheg Zhegju Zhag Uiversity of Wiscosi-Madiso Supplemetary Material S Notatio
More information1 Hash tables. 1.1 Implementation
Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationA Simplified Binet Formula for k-generalized Fibonacci Numbers
A Simplified Biet Formula for k-geeralized Fiboacci Numbers Gregory P. B. Dresde Departmet of Mathematics Washigto ad Lee Uiversity Lexigto, VA 440 dresdeg@wlu.edu Zhaohui Du Shaghai, Chia zhao.hui.du@gmail.com
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More information