Solving Continuous Linear Least-Squares Problems by Iterated Projection
|
|
- Cordelia Haynes
- 5 years ago
- Views:
Transcription
1 Solving Continuous Linear Least-Squares Problems by Iterated Projection by Ral Juengling Department o Computer Science, Portland State University PO Box 75 Portland, OR 977 USA juenglin@cs.pdx.edu Abstract I present a new divide-and-conquer algoritm or solving continuous linear least-squares problems. Te metod is applicable wen te column space o te linear system relating data to model parameters is translation invariant. Te central operation is a matrixvector product, wic maes te metod very easy to implement. Secondly, te structure o te computation suggests a straigtorward parallel implementation. A complexity analysis or sequential implementation sows tat te metod as te same asymptotic complexity as well-nown algoritms or discrete linear least-squares. For illustration we wor out te details or te problem o itting quadratic bivariate polynomials to a piecewise constant unction. Introduction Te linear least-squares problem is to determine te parameter values β or a unction tat minimizes te sum o squared dierences between data values {d i } and unction values {x i ; β} or corresponding points {x i }, m β = argmin β d i x i ; β. i= Te problem is a linear least-squares problem i te model is linear in te parameters, tat is, n x; β = = β x. A classical way o determining β goes as ollows:. Express in matrix orm, β = min d Aβ, β were A= x x n x x x n x x m x m n x m and d is te vector o data values.. Construct and solve te so called normal equations, A T Aβ = A T d. Assuming m > n more data values tan parameters and tat A as ull ran, te solution to may be written in terms o te pseudoinverse A + o A, β = A T A A T d = A + d. 5 Geometrically speaing, d = Aβ is te point in te range o A closest to d in te Euclidean norm. Te pseudoinverse projects d to te solution β, wic may be tougt o as te coordinates o d in te column space o A. In te ollowing we consider te continuous, linear version o problem, β = argmin β dx β x dx. 6 Ω
2 Section Note tat in te continuous setting we assume tere is a data unction dx, deined over some domain Ω. An example is single-band image data, wic is oten interpreted as a piecewise constant unction over te image domain []. Section introduces te ey ideas o te new metod and explains its prerequisites. Section gives te algoritm and discusses its computational merits. As an example, in Section 5 I wor out te elements o te algoritm or te problem o itting a bivariate quadratic polynomial. Te metod or solving 6 tat I call iterated projection wors or any dimension o Ω, but or te sae o clear exposition I assume Ω as dimension and Ω =Ω =[, ] [, ]. Linear Least-Squares by Iterated Projection Figure. Domain Ω wit coordinate origin at center a. Ω partitioned into our subdomains Ω, Ω, Ω, Ω in clocwise order b, next iner partition c, inest partition at wic te data unction d may be accurately represented over te subdomains d. A local coordinate system is centered on eac subdomain only sown or upper let subdomains in b and c. Recall tat te pseudoinverse in 5 projects te data directly onto a solution vector. Wit iterated projection we also obtain te solution by projection. However, instead o directly projecting te data we project solutions o intermediate least-squares problems. Imagine we partition Ω into our equally sized subdomains Ω,, Ω as sown in Figure b, and tat we are able to solve 6 or eac subdomain, wit solutions β,, β. Can we compute te solution β or Ω rom te our solutions β,, β or Ω,, Ω? As we will see below, te answer is yes, under certain circumstances. Wit problem 6 or subdomain Ω i we mean tat te integral in 6 is over Ω i instead o Ω, and tat te coordinates are wit respect to a coordinate system centered on Ω i c. Figure b. But centering te coordinate systems wit respect to eac Ω i means te our integrals are over te same set o points Ω = [ /, /] [ /, /]. Tese are important aspects o iterated projection: te problems corresponding to te our subdomains are our instances o te same problem only te data term in 6 is dierent, as well as similar to te original problem Ω =Ω instead o Ω =Ω. A ew more bits o notation are necessary to deine te intermediate problems more precisely. We denote te coordinates wit respect to Ω o te our subdomain s coordinate origins by x,, x and te solutions corresponding to te our subdomain problems by β,, β, respectively. In te continuous setting β are te coordinates o a unction closest to d in te space F o unctions over Ω tat is spanned by { Ω } = n. Liewise, te solution β i or subdomain Ω i are te coordinates o a unction closest to x dx + x i, in te space F o unctions over Ω spanned by { Ω } = n. We can use te space F to construct anoter space o unctions over Ω, F = { : Ω R x x+x i F or all x Ω and i= }. Note tat F as dimension n, wereas F as dimension n. Te answer to our question is tat we can construct a solution β or 6 rom te solutions β i or te subdomains i F is a subspace o F, F F. 7
3 Linear Least-Squares by Iterated Projection. Te Subspace Property Wen does te subspace property 7 old or a set o unctions { }? Wat we must veriy is tat or any set o parameters β tere are parameters β,, β suc tat β x x i = β i, x, x Ω, i=,,. In oter words, relation 7 olds wen te space o unctions over R spanned by { } is translation-invariant. Examples are te spaces o polynomials o any inite order, or te space o unctions spanned by sine-cosine pairs wit same wave number.. Solving te Linear Least-Squares Problem We want to obtain te linear least-squares solution β on Ω = Ω rom te solutions β,, β or Ω s subdomains Ω,, Ω, respectively. As we said above, we obtain β by projecting te solutions β,, β. Let a unction F be parameterized by β R n, β T = β T β T β T T β, were β i corresponds to te part o covering subdomain Ω i, and let F be parameterized by γ R n. Te projector rom F onto F we see maps β to te vector β = argmin γ β i,l l x dx. γ x x i 9 i Ω l Assume, or te moment, we now te projector, a n n-matrix, and call it R. Using R, te solution to problem 6 is β = R β = β R, R, R, R, β β β R,i denotes te obvious n n submatrix o R. Equation expresses te central idea and operation o te iterated projection algoritm: obtain solutions to smaller versions o te linear least-squares problem irst, ten derive te solution to te original problem by a simple matrixvector product. Te solutions to te smaller problems are obtained in te same way, by projecting solutions or sub-subdomains, using a similar projection operator R /. Te resulting algoritm is recursive. Te recursion ends at subdomains over wic te data unction d is an element o te space spanned by { } over tat domain, or wen d may be approximated in tat space wit negligible error Figure d.. Te Projection Operator We now turn to te problem o determining te projection matrix R used in. Tere are dierent ways o deriving R. In tis section I describe one way, and in Section 5 I demonstrate anoter way by example. To begin note tat wen subspace property 7 olds, or every element β in te range o R tere is one vector β or wic 9 is exactly zero. I tis is te case, ten β and β =R β represent te same unction over Ω. We denote P te n n-matrix tat maps β representing a unction in F to te vector β representing te same unction in F, β = P β = P, P, P, P, β. Determining P or a particular set o unctions { } is just anoter way o conirming tat te subspace property 7 olds. P is relevant ere because concrete expressions or it are easy to derive, and we can express R in terms o P below.
4 Section Te second ingredient we need is te inner product on te parameter space o F tat corresponds to te least-squares Euclidean norm,, g = x gx dx Ω = β x γ l l x dx Ω l Ω Ω Ω n = β T Ω Ω Ω n n γ Ω n Ω n Ω n n = β T Q γ. Q is te Gram matrix o te unctions { }. Inner products corresponding to te subdomainunction spaces are deined in te same way, Ω Ω T Ω, g Ω = β i Ω Ω Ω n Ω n Ω n Ω n n γ i = β T i Q γ i. Ω n n Wit P and Q we can express te least-squares property 9, wic is te deining property o R, witout using integrals R β = argmin γ β i,l l x dx γ x x i i Ω l = argmin γ β i,l l x dx P γ i, x i Ω l = argmin γ β i P γ i T Q β i P γ i i Since te sum in is quadratic in γ we can set its gradient w.r.t γ to zero and solve or γ to ind te minimizer. Te result is te ollowing expression or R, R = P T,i Q P,i P T, Q P T, Q P T, Q P T, Q. 5 i= Computational Caracteristics Te ollowing algoritm Iterated_Projection_LSQ is a very simple instance o iterated projection; it requires tat te data unction is piecewise constant over square regions o a regular partition o Ω into m m = l l regions c. Figure d. Not included in te algoritm description is te construction o te projection matrices R. Tese are assumed to ave been precomputed or all relevant scales prior to calling Iterated_Projection_LSQ. Iterated_Projection_LSQD, Input. D: m m data matrix, : scale o domain Output. β : least-squares solution over domain Ω = [, ] [, ] Steps.. I m = :. β project constant unction dx =D, Section.. else:. partition D, D = D D D D
5 Computational Caracteristics 5 5. β i Iterated_Projection_LSQD i, /, or i =,, 6. β R β β β β Equation. Computational Complexity o Iterated Projection Algoritm Iterated_Projection_LSQ is ormulated or two-dimensional domains. However, adapting te algoritm to domains o dierent dimensionality is straigtorward and we discuss computational complexity or te general case d dimensions. In te general case D is assumed to ave m d entries and is partitioned into d parts in Step, te projection matrix R is o size n d n, te dept o te recursion l = log m is independent o d. We assume tat analytical expressions or R are available, tus, precomputing te projection matrices means evaluating tose expressions or all l scales. Assuming tat te amount o computation or evaluating eac R -entry is independent o, precomputing te projecting matrices requires Olog m d n operations and Olog m d n space. Computing β in Step. requires On operations per entry o D, or Om d n operations total. Computing β in Step 6. requires a matrix-vector product o O d n. Step 6 is carried out i= l d i = d l d times, resulting in a total operation count o O d logm n or Om d n or Step 6. Steps 5. and 6. togeter may be organized in a loop over all D i. In tat case te amount o space required by iterated projection in addition to te space required or te projection matrices is proportional to n and proportional to te dept o te recursion, Olog m n.. Comparison to Classical Linear Least-Squares Algoritms In discrete linear least-squares problems, te problem domain Ω as been abstracted away and its dimension usually plays no part in a complexity analysis. Instead, te complexity o tose algoritms are expressed in te size o matrix A in, say M n []. Setting M = m d or comparison, te cost o iterated projection is OM n d operations, and O log d M n units o space ignoring space required or entering te problem. A classical algoritm or solving te normal equations is Colesy actorization, Colesy_LSQA, d Input. A: M n model matrix, d: M data vector Output. β : least-squares solution argmin β Aβ d Steps.. Compute B = A T A. Compute y = A T d. Factorize B, B = R T R R upper triangular. Solve R T x= y 5. Solve R β = x Te costs o tese steps are, respectively, OM n, OM n, On, On, and On operations. In most situations were iterated projection is applicable we could carry out Step. and Step. o Colesy_LSQ in advance, and te asymptotic cost would be dominated by Step., OM n. Storing te Colesy actors requires On units o space. Oter classical algoritm s complexity caracteristics are essentielly te same [], Capter, and we conclude tat te asymptotic computational cost o iterated projection is te same as te cost o well-nown algoritms or discrete linear least-squares.
6 6 Section 5 Discussion I ave presented a new idea, iterated projection, or computing te solution to continuous linear least-squares problems. Te idea is to project te data wic is tougt o as a unction into a sequence o unction spaces o decreasing dimensionality. Te inal projection in tis sequence is into te space in wic te solution is sougt. As discussed in Section., iterated projection is asymptotically as eicient as oter linear least-squares algoritms. However, Iterated_Projection_LSQ is a striingly simple algoritm, consisting essentially o a single operation te matrix-vector product in Step 6 and is arguably easier to accelerate by special ardware. Secondly, te way te computation is organized in Iterated_Projection_LSQ maes it easy to distribute te wor among multiple processors. Classical linear least-squares algoritms and iterated projection do not solve exactly te same problem. Te ormer solve te problem Find β suc tat d Aβ is minimal or β = β, 6 iterated projection solves te problem wit linear in β Find β suc tat dx β x dx is minimal or β = β. 7 Ω In practice problem 7 is oten approximated by a problem o te orm 6, obtained troug sampling d and at selected points {x i } c. to in Section. In suc situations, and wen te subspace property 7 olds, iterated projection is a compelling alternative to discrete linear least-squares algoritms. It may give exact solutions except or rounding errors, and it may require less computation wen d is o appropriate orm e.g., unctions in inite-element spaces; Step in Iterated_Projection_LSQ needs to be adapted accordingly. Anoter interesting aspect o iterated projection is tat, since no linear system needs to be solved, te problems o underdetermined or singular systems do not occur. Tis recommends iterated projection or problems lie least-squares based image reconstruction and segmentation [, ], were small regions oten lead to underdetermined or singular least-squares problems. 5 Appendix Iterated Projection wit Quadratic Polynomials In tis appendix we derive te projection matrix R or problem 7 wit { } given as x = x = x x = x x = x 5 x = x x 6 x = x. At irst we veriy tat te space spanned by,, 6 is translation invariant. Let β represent any suc unction, x; β = 6 = β x, x be te origin o anoter coordinate system, and x ; β denote te representation wit respect to te coordinate system centered at x. Function is te same as i teir values and tat o all teir derivatives are identical at any point. Coose x as tat point and write down te identities to obtain β in terms o β and x. β = β + β x + β x + β x + β5 x x + β 6 x β = β + β x + β 5 x,,,,,, and,, respec- To derive matrix P in we set x to tively, and obtain
7 Appendix Iterated Projection wit Quadratic Polynomials 7 P, = P, = P, = P, =. Matrix Q in can be seen to be Q = Tese are te ingredients needed to derive R via Deriving R by QR Factorization One algoritm or solving least-squares problem is by QR actorization. It consists o computing te reduced QR actorization o A in, A = Q R, by wic we obtain an ortonormal basis or te column space o A. Next, te data vector is expanded in tat ortonormal basis, and, inally, an expansion in te basis o interest te columns o A is obtained by solving a triangular system. I now sow ow to derive R by applying essentially tis algoritm, but in te continuous setting. Te polynomials do not orm an ortogonal basis wit respect to te inner product, oterwise Q in 9 were diagonal. An ortornomal basis or te space spanned by are te Legendre polynomials up to order two. Deined over Ω =[, ] [, ], tese are φ x = φ x = x φ x = x φ x = 5 x φ 5 x = x x φ 6 x = 5 x. Te QR actorization o A= 5 6 over Ω is 5 6 = φ φ φ φ φ 5 φ 6 = Φ C 5 5
8 Section Te product R β may now be expressed as R β = C Φ T 5 6 dx β i i= Ω i φ, / φ, / φ, 6 / = C φ, / φ, / φ 5, 6 / β i i= φ 6, / φ 6, / φ 6, 6 / = C β, R, R, R, R, ence R = C R, C R, C R, C R,. Evaluating te integrals in R,i we get R, = R, = R, = R, = Multiplying R,i by C we inally arrive at te our components o R, R, = R, = R, = R, = A Matlab implementation o iterated projection based on tese results is available or download rom tp://tp.cs.pdx.edu/pub/juenglin/iterated_projection/examples.tar. Bibliograpy [] P. J. Besl and R. C. Jain. Segmentation troug variable-order surace itting. IEEE Transactions on Pattern Analysis and Macine Intelligence, :67 9, Marc 9. [] T. Kanungo, B. Dom, W. Niblac, and D. Steele. A ast algoritm or MDL-based multi-band image segmentation. In Proceedings o te Conerence on Computer Vision and Pattern Recognition, pages 69 66, Los Alamitos, CA, USA, 99. IEEE Computer Society Press. [] Y. G. Leclerc. Constructing simple stable descriptions or image partitioning. International Journal o Computer Vision, :7, May 99. [] L. N. Treeten and D. Bau. Numerical Linear Algebra. SIAM, Piladelpia, PA, 997.
Taylor Series and the Mean Value Theorem of Derivatives
1 - Taylor Series and te Mean Value Teorem o Derivatives Te numerical solution o engineering and scientiic problems described by matematical models oten requires solving dierential equations. Dierential
More informationUNIT #6 EXPONENTS, EXPONENTS, AND MORE EXPONENTS REVIEW QUESTIONS
Answer Key Name: Date: UNIT # EXPONENTS, EXPONENTS, AND MORE EXPONENTS REVIEW QUESTIONS Part I Questions. Te epression 0 can be simpliied to () () 0 0. Wic o te ollowing is equivalent to () () 8 8? 8.
More informationNumerical Differentiation
Numerical Differentiation Finite Difference Formulas for te first derivative (Using Taylor Expansion tecnique) (section 8.3.) Suppose tat f() = g() is a function of te variable, and tat as 0 te function
More informationContinuity and Differentiability
Continuity and Dierentiability Tis capter requires a good understanding o its. Te concepts o continuity and dierentiability are more or less obvious etensions o te concept o its. Section - INTRODUCTION
More informationMATH745 Fall MATH745 Fall
MATH745 Fall 5 MATH745 Fall 5 INTRODUCTION WELCOME TO MATH 745 TOPICS IN NUMERICAL ANALYSIS Instructor: Dr Bartosz Protas Department of Matematics & Statistics Email: bprotas@mcmasterca Office HH 36, Ext
More informationExam 1 Review Solutions
Exam Review Solutions Please also review te old quizzes, and be sure tat you understand te omework problems. General notes: () Always give an algebraic reason for your answer (graps are not sufficient),
More information1. Questions (a) through (e) refer to the graph of the function f given below. (A) 0 (B) 1 (C) 2 (D) 4 (E) does not exist
Mat 1120 Calculus Test 2. October 18, 2001 Your name Te multiple coice problems count 4 points eac. In te multiple coice section, circle te correct coice (or coices). You must sow your work on te oter
More information158 Calculus and Structures
58 Calculus and Structures CHAPTER PROPERTIES OF DERIVATIVES AND DIFFERENTIATION BY THE EASY WAY. Calculus and Structures 59 Copyrigt Capter PROPERTIES OF DERIVATIVES. INTRODUCTION In te last capter you
More informationIntroduction to Derivatives
Introduction to Derivatives 5-Minute Review: Instantaneous Rates and Tangent Slope Recall te analogy tat we developed earlier First we saw tat te secant slope of te line troug te two points (a, f (a))
More informationChapter 2 Limits and Continuity. Section 2.1 Rates of Change and Limits (pp ) Section Quick Review 2.1
Section. 6. (a) N(t) t (b) days: 6 guppies week: 7 guppies (c) Nt () t t t ln ln t ln ln ln t 8. 968 Tere will be guppies ater ln 8.968 days, or ater nearly 9 days. (d) Because it suggests te number o
More informationEfficient algorithms for for clone items detection
Efficient algoritms for for clone items detection Raoul Medina, Caroline Noyer, and Olivier Raynaud Raoul Medina, Caroline Noyer and Olivier Raynaud LIMOS - Université Blaise Pascal, Campus universitaire
More informationDifferentiation Rules c 2002 Donald Kreider and Dwight Lahr
Dierentiation Rules c 00 Donal Kreier an Dwigt Lar Te Power Rule is an example o a ierentiation rule. For unctions o te orm x r, were r is a constant real number, we can simply write own te erivative rater
More informationMAT 145. Type of Calculator Used TI-89 Titanium 100 points Score 100 possible points
MAT 15 Test #2 Name Solution Guide Type of Calculator Used TI-89 Titanium 100 points Score 100 possible points Use te grap of a function sown ere as you respond to questions 1 to 8. 1. lim f (x) 0 2. lim
More informationSolutions to the Multivariable Calculus and Linear Algebra problems on the Comprehensive Examination of January 31, 2014
Solutions to te Multivariable Calculus and Linear Algebra problems on te Compreensive Examination of January 3, 24 Tere are 9 problems ( points eac, totaling 9 points) on tis portion of te examination.
More informationHOMEWORK HELP 2 FOR MATH 151
HOMEWORK HELP 2 FOR MATH 151 Here we go; te second round of omework elp. If tere are oters you would like to see, let me know! 2.4, 43 and 44 At wat points are te functions f(x) and g(x) = xf(x)continuous,
More informationFunction Composition and Chain Rules
Function Composition and s James K. Peterson Department of Biological Sciences and Department of Matematical Sciences Clemson University Marc 8, 2017 Outline 1 Function Composition and Continuity 2 Function
More information3.4 Worksheet: Proof of the Chain Rule NAME
Mat 1170 3.4 Workseet: Proof of te Cain Rule NAME Te Cain Rule So far we are able to differentiate all types of functions. For example: polynomials, rational, root, and trigonometric functions. We are
More informationConsider a function f we ll specify which assumptions we need to make about it in a minute. Let us reformulate the integral. 1 f(x) dx.
Capter 2 Integrals as sums and derivatives as differences We now switc to te simplest metods for integrating or differentiating a function from its function samples. A careful study of Taylor expansions
More informationDigital Filter Structures
Digital Filter Structures Te convolution sum description of an LTI discrete-time system can, in principle, be used to implement te system For an IIR finite-dimensional system tis approac is not practical
More information232 Calculus and Structures
3 Calculus and Structures CHAPTER 17 JUSTIFICATION OF THE AREA AND SLOPE METHODS FOR EVALUATING BEAMS Calculus and Structures 33 Copyrigt Capter 17 JUSTIFICATION OF THE AREA AND SLOPE METHODS 17.1 THE
More information5.1 We will begin this section with the definition of a rational expression. We
Basic Properties and Reducing to Lowest Terms 5.1 We will begin tis section wit te definition of a rational epression. We will ten state te two basic properties associated wit rational epressions and go
More informationSolve exponential equations in one variable using a variety of strategies. LEARN ABOUT the Math. What is the half-life of radon?
8.5 Solving Exponential Equations GOAL Solve exponential equations in one variable using a variety of strategies. LEARN ABOUT te Mat All radioactive substances decrease in mass over time. Jamie works in
More informationPolynomial Interpolation
Capter 4 Polynomial Interpolation In tis capter, we consider te important problem of approximatinga function fx, wose values at a set of distinct points x, x, x,, x n are known, by a polynomial P x suc
More informationContinuity and Differentiability Worksheet
Continuity and Differentiability Workseet (Be sure tat you can also do te grapical eercises from te tet- Tese were not included below! Typical problems are like problems -3, p. 6; -3, p. 7; 33-34, p. 7;
More informationMATH1901 Differential Calculus (Advanced)
MATH1901 Dierential Calculus (Advanced) Capter 3: Functions Deinitions : A B A and B are sets assigns to eac element in A eactl one element in B A is te domain o te unction B is te codomain o te unction
More informationf a h f a h h lim lim
Te Derivative Te derivative of a function f at a (denoted f a) is f a if tis it exists. An alternative way of defining f a is f a x a fa fa fx fa x a Note tat te tangent line to te grap of f at te point
More informationComputer Derivations of Numerical Differentiation Formulae. Int. J. of Math. Education in Sci. and Tech., V 34, No 2 (March-April 2003), pp
Computer Derivations o Numerical Dierentiation Formulae By Jon H. Matews Department o Matematics Caliornia State University Fullerton USA Int. J. o Mat. Education in Sci. and Tec. V No (Marc-April ) pp.8-87.
More informationPre-Calculus Review Preemptive Strike
Pre-Calculus Review Preemptive Strike Attaced are some notes and one assignment wit tree parts. Tese are due on te day tat we start te pre-calculus review. I strongly suggest reading troug te notes torougly
More information1watt=1W=1kg m 2 /s 3
Appendix A Matematics Appendix A.1 Units To measure a pysical quantity, you need a standard. Eac pysical quantity as certain units. A unit is just a standard we use to compare, e.g. a ruler. In tis laboratory
More informationPolynomial Interpolation
Capter 4 Polynomial Interpolation In tis capter, we consider te important problem of approximating a function f(x, wose values at a set of distinct points x, x, x 2,,x n are known, by a polynomial P (x
More informationSymmetry Labeling of Molecular Energies
Capter 7. Symmetry Labeling of Molecular Energies Notes: Most of te material presented in tis capter is taken from Bunker and Jensen 1998, Cap. 6, and Bunker and Jensen 2005, Cap. 7. 7.1 Hamiltonian Symmetry
More information4. The slope of the line 2x 7y = 8 is (a) 2/7 (b) 7/2 (c) 2 (d) 2/7 (e) None of these.
Mat 11. Test Form N Fall 016 Name. Instructions. Te first eleven problems are wort points eac. Te last six problems are wort 5 points eac. For te last six problems, you must use relevant metods of algebra
More informationChapter 1D - Rational Expressions
- Capter 1D Capter 1D - Rational Expressions Definition of a Rational Expression A rational expression is te quotient of two polynomials. (Recall: A function px is a polynomial in x of degree n, if tere
More informationHydraulic validation of the LHC cold mass heat exchanger tube.
Hydraulic validation o te LHC cold mass eat excanger tube. LHC Project Note 155 1998-07-22 (pilippe.provenaz@cern.c) Pilippe PROVENAZ / LHC-ACR Division Summary Te knowledge o te elium mass low vs. te
More informationThe Verlet Algorithm for Molecular Dynamics Simulations
Cemistry 380.37 Fall 2015 Dr. Jean M. Standard November 9, 2015 Te Verlet Algoritm for Molecular Dynamics Simulations Equations of motion For a many-body system consisting of N particles, Newton's classical
More informationContinuity and Differentiability of the Trigonometric Functions
[Te basis for te following work will be te definition of te trigonometric functions as ratios of te sides of a triangle inscribed in a circle; in particular, te sine of an angle will be defined to be te
More informationSECTION 3.2: DERIVATIVE FUNCTIONS and DIFFERENTIABILITY
(Section 3.2: Derivative Functions and Differentiability) 3.2.1 SECTION 3.2: DERIVATIVE FUNCTIONS and DIFFERENTIABILITY LEARNING OBJECTIVES Know, understand, and apply te Limit Definition of te Derivative
More informationMaterial for Difference Quotient
Material for Difference Quotient Prepared by Stepanie Quintal, graduate student and Marvin Stick, professor Dept. of Matematical Sciences, UMass Lowell Summer 05 Preface Te following difference quotient
More information7 Semiparametric Methods and Partially Linear Regression
7 Semiparametric Metods and Partially Linear Regression 7. Overview A model is called semiparametric if it is described by and were is nite-dimensional (e.g. parametric) and is in nite-dimensional (nonparametric).
More informationQuantum Numbers and Rules
OpenStax-CNX module: m42614 1 Quantum Numbers and Rules OpenStax College Tis work is produced by OpenStax-CNX and licensed under te Creative Commons Attribution License 3.0 Abstract Dene quantum number.
More informationChapter 4: Numerical Methods for Common Mathematical Problems
1 Capter 4: Numerical Metods for Common Matematical Problems Interpolation Problem: Suppose we ave data defined at a discrete set of points (x i, y i ), i = 0, 1,..., N. Often it is useful to ave a smoot
More informationCombining functions: algebraic methods
Combining functions: algebraic metods Functions can be added, subtracted, multiplied, divided, and raised to a power, just like numbers or algebra expressions. If f(x) = x 2 and g(x) = x + 2, clearly f(x)
More informationPolynomials 3: Powers of x 0 + h
near small binomial Capter 17 Polynomials 3: Powers of + Wile it is easy to compute wit powers of a counting-numerator, it is a lot more difficult to compute wit powers of a decimal-numerator. EXAMPLE
More information1 Calculus. 1.1 Gradients and the Derivative. Q f(x+h) f(x)
Calculus. Gradients and te Derivative Q f(x+) δy P T δx R f(x) 0 x x+ Let P (x, f(x)) and Q(x+, f(x+)) denote two points on te curve of te function y = f(x) and let R denote te point of intersection of
More informationMidterm #1B. x 8 < < x 8 < 11 3 < x < x > x < 5 or 3 2x > 5 2x < 8 2x > 2
Mat 30 College Algebra Februar 2, 2016 Midterm #1B Name: Answer Ke David Arnold Instructions. ( points) For eac o te ollowing questions, select te best answer and darken te corresponding circle on our
More informationThe total error in numerical differentiation
AMS 147 Computational Metods and Applications Lecture 08 Copyrigt by Hongyun Wang, UCSC Recap: Loss of accuracy due to numerical cancellation A B 3, 3 ~10 16 In calculating te difference between A and
More informationThese errors are made from replacing an infinite process by finite one.
Introduction :- Tis course examines problems tat can be solved by metods of approximation, tecniques we call numerical metods. We begin by considering some of te matematical and computational topics tat
More informationNumerical Experiments Using MATLAB: Superconvergence of Nonconforming Finite Element Approximation for Second-Order Elliptic Problems
Applied Matematics, 06, 7, 74-8 ttp://wwwscirporg/journal/am ISSN Online: 5-7393 ISSN Print: 5-7385 Numerical Experiments Using MATLAB: Superconvergence of Nonconforming Finite Element Approximation for
More informationBob Brown Math 251 Calculus 1 Chapter 3, Section 1 Completed 1 CCBC Dundalk
Bob Brown Mat 251 Calculus 1 Capter 3, Section 1 Completed 1 Te Tangent Line Problem Te idea of a tangent line first arises in geometry in te context of a circle. But before we jump into a discussion of
More informationPrecalculus Test 2 Practice Questions Page 1. Note: You can expect other types of questions on the test than the ones presented here!
Precalculus Test 2 Practice Questions Page Note: You can expect oter types of questions on te test tan te ones presented ere! Questions Example. Find te vertex of te quadratic f(x) = 4x 2 x. Example 2.
More informationExercises for numerical differentiation. Øyvind Ryan
Exercises for numerical differentiation Øyvind Ryan February 25, 2013 1. Mark eac of te following statements as true or false. a. Wen we use te approximation f (a) (f (a +) f (a))/ on a computer, we can
More informationGeneric maximum nullity of a graph
Generic maximum nullity of a grap Leslie Hogben Bryan Sader Marc 5, 2008 Abstract For a grap G of order n, te maximum nullity of G is defined to be te largest possible nullity over all real symmetric n
More informationHow to Find the Derivative of a Function: Calculus 1
Introduction How to Find te Derivative of a Function: Calculus 1 Calculus is not an easy matematics course Te fact tat you ave enrolled in suc a difficult subject indicates tat you are interested in te
More information1. Which one of the following expressions is not equal to all the others? 1 C. 1 D. 25x. 2. Simplify this expression as much as possible.
004 Algebra Pretest answers and scoring Part A. Multiple coice questions. Directions: Circle te letter ( A, B, C, D, or E ) net to te correct answer. points eac, no partial credit. Wic one of te following
More informationPhase space in classical physics
Pase space in classical pysics Quantum mecanically, we can actually COU te number of microstates consistent wit a given macrostate, specified (for example) by te total energy. In general, eac microstate
More informationMathematics 5 Worksheet 11 Geometry, Tangency, and the Derivative
Matematics 5 Workseet 11 Geometry, Tangency, and te Derivative Problem 1. Find te equation of a line wit slope m tat intersects te point (3, 9). Solution. Te equation for a line passing troug a point (x
More informationLab 6 Derivatives and Mutant Bacteria
Lab 6 Derivatives and Mutant Bacteria Date: September 27, 20 Assignment Due Date: October 4, 20 Goal: In tis lab you will furter explore te concept of a derivative using R. You will use your knowledge
More information2.8 The Derivative as a Function
.8 Te Derivative as a Function Typically, we can find te derivative of a function f at many points of its domain: Definition. Suppose tat f is a function wic is differentiable at every point of an open
More informationMath 1210 Midterm 1 January 31st, 2014
Mat 110 Midterm 1 January 1st, 01 Tis exam consists of sections, A and B. Section A is conceptual, wereas section B is more computational. Te value of every question is indicated at te beginning of it.
More information1 The concept of limits (p.217 p.229, p.242 p.249, p.255 p.256) 1.1 Limits Consider the function determined by the formula 3. x since at this point
MA00 Capter 6 Calculus and Basic Linear Algebra I Limits, Continuity and Differentiability Te concept of its (p.7 p.9, p.4 p.49, p.55 p.56). Limits Consider te function determined by te formula f Note
More information5 Ordinary Differential Equations: Finite Difference Methods for Boundary Problems
5 Ordinary Differential Equations: Finite Difference Metods for Boundary Problems Read sections 10.1, 10.2, 10.4 Review questions 10.1 10.4, 10.8 10.9, 10.13 5.1 Introduction In te previous capters we
More informationDerivatives. By: OpenStaxCollege
By: OpenStaxCollege Te average teen in te United States opens a refrigerator door an estimated 25 times per day. Supposedly, tis average is up from 10 years ago wen te average teenager opened a refrigerator
More informationlecture 26: Richardson extrapolation
43 lecture 26: Ricardson extrapolation 35 Ricardson extrapolation, Romberg integration Trougout numerical analysis, one encounters procedures tat apply some simple approximation (eg, linear interpolation)
More informationPreface. Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.
Preface Here are my online notes for my course tat I teac ere at Lamar University. Despite te fact tat tese are my class notes, tey sould be accessible to anyone wanting to learn or needing a refreser
More informationThe derivative function
Roberto s Notes on Differential Calculus Capter : Definition of derivative Section Te derivative function Wat you need to know already: f is at a point on its grap and ow to compute it. Wat te derivative
More informationPractice Problem Solutions: Exam 1
Practice Problem Solutions: Exam 1 1. (a) Algebraic Solution: Te largest term in te numerator is 3x 2, wile te largest term in te denominator is 5x 2 3x 2 + 5. Tus lim x 5x 2 2x 3x 2 x 5x 2 = 3 5 Numerical
More informationMTH 119 Pre Calculus I Essex County College Division of Mathematics Sample Review Questions 1 Created April 17, 2007
MTH 9 Pre Calculus I Essex County College Division of Matematics Sample Review Questions Created April 7, 007 At Essex County College you sould be prepared to sow all work clearly and in order, ending
More information5.1 The derivative or the gradient of a curve. Definition and finding the gradient from first principles
Capter 5: Dierentiation In tis capter, we will study: 51 e derivative or te gradient o a curve Deinition and inding te gradient ro irst principles 5 Forulas or derivatives 5 e equation o te tangent line
More informationMath 1241 Calculus Test 1
February 4, 2004 Name Te first nine problems count 6 points eac and te final seven count as marked. Tere are 120 points available on tis test. Multiple coice section. Circle te correct coice(s). You do
More information4.2 - Richardson Extrapolation
. - Ricardson Extrapolation. Small-O Notation: Recall tat te big-o notation used to define te rate of convergence in Section.: Definition Let x n n converge to a number x. Suppose tat n n is a sequence
More informationAMS 147 Computational Methods and Applications Lecture 09 Copyright by Hongyun Wang, UCSC. Exact value. Effect of round-off error.
Lecture 09 Copyrigt by Hongyun Wang, UCSC Recap: Te total error in numerical differentiation fl( f ( x + fl( f ( x E T ( = f ( x Numerical result from a computer Exact value = e + f x+ Discretization error
More informationMATH1151 Calculus Test S1 v2a
MATH5 Calculus Test 8 S va January 8, 5 Tese solutions were written and typed up by Brendan Trin Please be etical wit tis resource It is for te use of MatSOC members, so do not repost it on oter forums
More informationLecture 15. Interpolation II. 2 Piecewise polynomial interpolation Hermite splines
Lecture 5 Interpolation II Introduction In te previous lecture we focused primarily on polynomial interpolation of a set of n points. A difficulty we observed is tat wen n is large, our polynomial as to
More informationOrder of Accuracy. ũ h u Ch p, (1)
Order of Accuracy 1 Terminology We consider a numerical approximation of an exact value u. Te approximation depends on a small parameter, wic can be for instance te grid size or time step in a numerical
More informationRecall from our discussion of continuity in lecture a function is continuous at a point x = a if and only if
Computational Aspects of its. Keeping te simple simple. Recall by elementary functions we mean :Polynomials (including linear and quadratic equations) Eponentials Logaritms Trig Functions Rational Functions
More informationDifferentiation in higher dimensions
Capter 2 Differentiation in iger dimensions 2.1 Te Total Derivative Recall tat if f : R R is a 1-variable function, and a R, we say tat f is differentiable at x = a if and only if te ratio f(a+) f(a) tends
More informationLIMITS AND DERIVATIVES CONDITIONS FOR THE EXISTENCE OF A LIMIT
LIMITS AND DERIVATIVES Te limit of a function is defined as te value of y tat te curve approaces, as x approaces a particular value. Te limit of f (x) as x approaces a is written as f (x) approaces, as
More informationNew Streamfunction Approach for Magnetohydrodynamics
New Streamfunction Approac for Magnetoydrodynamics Kab Seo Kang Brooaven National Laboratory, Computational Science Center, Building 63, Room, Upton NY 973, USA. sang@bnl.gov Summary. We apply te finite
More informationQuantum Mechanics Chapter 1.5: An illustration using measurements of particle spin.
I Introduction. Quantum Mecanics Capter.5: An illustration using measurements of particle spin. Quantum mecanics is a teory of pysics tat as been very successful in explaining and predicting many pysical
More informationAnalytic Functions. Differentiable Functions of a Complex Variable
Analytic Functions Differentiable Functions of a Complex Variable In tis capter, we sall generalize te ideas for polynomials power series of a complex variable we developed in te previous capter to general
More informationA SHORT INTRODUCTION TO BANACH LATTICES AND
CHAPTER A SHORT INTRODUCTION TO BANACH LATTICES AND POSITIVE OPERATORS In tis capter we give a brief introduction to Banac lattices and positive operators. Most results of tis capter can be found, e.g.,
More information1 Solutions to the in class part
NAME: Solutions to te in class part. Te grap of a function f is given. Calculus wit Analytic Geometry I Exam, Friday, August 30, 0 SOLUTIONS (a) State te value of f(). (b) Estimate te value of f( ). (c)
More informationE E B B. over U is a full subcategory of the fiber of C,D over U. Given [B, B ], and h=θ over V, the Cartesian arrow M=f
ppendix : ibrational teory o L euivalence E E onsider ibrations P, P, and te category Fib[,] o all maps E @E 2 =(, ):@ :: P { P 2 () @ o ibrations; Fib[,] is a ull subcategory o [,] ; see [3]. Fib[,] is
More information2.1 THE DEFINITION OF DERIVATIVE
2.1 Te Derivative Contemporary Calculus 2.1 THE DEFINITION OF DERIVATIVE 1 Te grapical idea of a slope of a tangent line is very useful, but for some uses we need a more algebraic definition of te derivative
More informationConductance from Transmission Probability
Conductance rom Transmission Probability Kelly Ceung Department o Pysics & Astronomy University o Britis Columbia Vancouver, BC. Canada, V6T1Z1 (Dated: November 5, 005). ntroduction For large conductors,
More information2.11 That s So Derivative
2.11 Tat s So Derivative Introduction to Differential Calculus Just as one defines instantaneous velocity in terms of average velocity, we now define te instantaneous rate of cange of a function at a point
More informationParameter Fitted Scheme for Singularly Perturbed Delay Differential Equations
International Journal of Applied Science and Engineering 2013. 11, 4: 361-373 Parameter Fitted Sceme for Singularly Perturbed Delay Differential Equations Awoke Andargiea* and Y. N. Reddyb a b Department
More informationMath Spring 2013 Solutions to Assignment # 3 Completion Date: Wednesday May 15, (1/z) 2 (1/z 1) 2 = lim
Mat 311 - Spring 013 Solutions to Assignment # 3 Completion Date: Wednesday May 15, 013 Question 1. [p 56, #10 (a)] 4z Use te teorem of Sec. 17 to sow tat z (z 1) = 4. We ave z 4z (z 1) = z 0 4 (1/z) (1/z
More informationSECTION 1.10: DIFFERENCE QUOTIENTS LEARNING OBJECTIVES
(Section.0: Difference Quotients).0. SECTION.0: DIFFERENCE QUOTIENTS LEARNING OBJECTIVES Define average rate of cange (and average velocity) algebraically and grapically. Be able to identify, construct,
More informationMAT244 - Ordinary Di erential Equations - Summer 2016 Assignment 2 Due: July 20, 2016
MAT244 - Ordinary Di erential Equations - Summer 206 Assignment 2 Due: July 20, 206 Full Name: Student #: Last First Indicate wic Tutorial Section you attend by filling in te appropriate circle: Tut 0
More informationOn One Justification on the Use of Hybrids for the Solution of First Order Initial Value Problems of Ordinary Differential Equations
Pure and Applied Matematics Journal 7; 6(5: 74 ttp://wwwsciencepublisinggroupcom/j/pamj doi: 648/jpamj765 ISSN: 6979 (Print; ISSN: 698 (Online On One Justiication on te Use o Hybrids or te Solution o First
More informationLecture XVII. Abstract We introduce the concept of directional derivative of a scalar function and discuss its relation with the gradient operator.
Lecture XVII Abstract We introduce te concept of directional derivative of a scalar function and discuss its relation wit te gradient operator. Directional derivative and gradient Te directional derivative
More informationSimulation and verification of a plate heat exchanger with a built-in tap water accumulator
Simulation and verification of a plate eat excanger wit a built-in tap water accumulator Anders Eriksson Abstract In order to test and verify a compact brazed eat excanger (CBE wit a built-in accumulation
More informationMTH-112 Quiz 1 Name: # :
MTH- Quiz Name: # : Please write our name in te provided space. Simplif our answers. Sow our work.. Determine weter te given relation is a function. Give te domain and range of te relation.. Does te equation
More informationGradient Descent etc.
1 Gradient Descent etc EE 13: Networked estimation and control Prof Kan) I DERIVATIVE Consider f : R R x fx) Te derivative is defined as d fx) = lim dx fx + ) fx) Te cain rule states tat if d d f gx) )
More informationDIGRAPHS FROM POWERS MODULO p
DIGRAPHS FROM POWERS MODULO p Caroline Luceta Box 111 GCC, 100 Campus Drive, Grove City PA 1617 USA Eli Miller PO Box 410, Sumneytown, PA 18084 USA Clifford Reiter Department of Matematics, Lafayette College,
More informationEssential Microeconomics : EQUILIBRIUM AND EFFICIENCY WITH PRODUCTION Key ideas: Walrasian equilibrium, first and second welfare theorems
Essential Microeconomics -- 5.2: EQUILIBRIUM AND EFFICIENCY WITH PRODUCTION Key ideas: Walrasian equilibrium, irst and second welare teorems A general model 2 First welare Teorem 7 Second welare teorem
More informationMATHEMATICS FOR ENGINEERING DIFFERENTIATION TUTORIAL 1 - BASIC DIFFERENTIATION
MATHEMATICS FOR ENGINEERING DIFFERENTIATION TUTORIAL 1 - BASIC DIFFERENTIATION Tis tutorial is essential pre-requisite material for anyone stuing mecanical engineering. Tis tutorial uses te principle of
More informationGeneral Solution of the Stress Potential Function in Lekhnitskii s Elastic Theory for Anisotropic and Piezoelectric Materials
dv. Studies Teor. Pys. Vol. 007 no. 8 7 - General Solution o te Stress Potential Function in Lenitsii s Elastic Teory or nisotropic and Pieoelectric Materials Zuo-en Ou StateKey Laboratory o Explosion
More information(a) At what number x = a does f have a removable discontinuity? What value f(a) should be assigned to f at x = a in order to make f continuous at a?
Solutions to Test 1 Fall 016 1pt 1. Te grap of a function f(x) is sown at rigt below. Part I. State te value of eac limit. If a limit is infinite, state weter it is or. If a limit does not exist (but is
More information