Introduction to Machine Learning. Recitation 8. w 2, b 2. w 1, b 1. z 0 z 1. The function we want to minimize is the loss over all examples: f =
|
|
- Marian Peters
- 5 years ago
- Views:
Transcription
1 Introduction to Macine Learning Lecturer: Regev Scweiger Recitation 8 Fall Semester Scribe: Regev Scweiger 8.1 Backpropagation We will develop and review te backpropagation algoritm for neural networks. In order to ave a concrete example, we will focus on te coice of te sigmoid (i.e., (z = 1/(1 + e z, wit te log loss function (i.e., l(y, ŷ = y log ŷ (1 y log(1 ŷ, but of course, te derivation olds in general Two layers, one node We will start from te simplest (interesting example and do it very explicitly. Suppose we are given a sample x 1,..., x m wit labels y 1,..., y m { 1, 1}. Te simplest is two layers, wit one node in eac: w 1, b 1 w 2, b 2 z 0 z 1 ŷ = z 2 Te function we want to minimize is te loss over all examples: f = m l(y i, z 2 (x i i=1 We take a gradient descent approac. At eac iteration, we ave our current guesses for te best parameter values: w 1, b 1, w 2, b 2, and we wis to calculate te gradient at tat point, e.g: f f w 1 and f w1 = w 1,b 1 = b 1, b 1 and f w1 = w 1,b 1 = b 1, w 2 and w1 = w 1,b 1 = b 1, b 2 w1 = w 1,b 1 = b 1, First, since derivative is linear, we can focus only on te loss over a single sample, and set y = y i, z 0 = x i. and, e.g.: f w 1 = m i=1 w 1 l(y, z 2 (z 0 1
2 2 Lecture 8 We proceed as if tere is a single sample. To continue, we will benefit by introducing convenient notation (see also lesson scribe and drawing it as well. Define by v 1 = z 0 w 1 +b 1, v 2 = z 1 w 2 +b 2 te linear combinations used in te calculations of z 1, z 2 (respectively. Namely, z 1 = (v 1, z 2 = (v 2. We will add tese to te grap above, tat will now sow te explicit calculation we make. We will also sow te loss function, since it is added to te entire calculation. w 1, b 1 w 2, b 2 y z 0 v 1 z 1 v 2 z 2 l(y, z 2 Derivative w.r.t w 2, b 2 Let first recall te cain rule. Suppose we ave tree variables/functions x, y, z, and z = z(y, y = y(x, tus z = z(y(x. Say we want to evaluate te derivative of z at a point x. Ten, te cain rule says: z x = z x= x y y y=y( x x We want to calculate l(y, z 2 (z 0 w 2 x= x w1 = w 1,b 1 = b 1, te dependency on te current point is idden in z 2. We will denote by ṽ i te evaluation of v i on te specific point we are at, e.g., ṽ 1 = z 0 w 1 + b 1. Similarly, z i = (ṽ i. Following te cain rule and te drawing above, we can see tat l(y, z 2 (z 0 w 2 w1 = w 1,b 1 = b 1, = l(y, z 2 z 2 z 2 z2 = z 2 v 2 v 2 w 2 z 1 = z 1, Te left expression is te derivative of te loss function wrt to te estimate. In our case, it is te log loss, evaluated at te current point, tat is: l(y, z 2 z 2 = y/ z 2 + (1 y/(1 z 2 z2 = z 2 Te middle expression can be solved by te useful identity (exercise (v = (v(1 (v, to give: z 2 v 2 = (ṽ 2 (1 (ṽ 2 = z 2 (1 z 2
3 8.1. BACKPROPAGATION 3 Te rigt expression is simply v 2 = z 1 w 2 z 1 = z 1, We can do te same for /b 2, we would get te same calculation, wit te coefficient 1 instead of z 1 - indeed, we can tink about it as anoter input of constant 1 being input into eac one of te nodes. Derivative w.r.t w 1, b 1 Wat about te derivative wrt w 1 (and b 1? Following te cain rule (and te drawing again, l(y, z 2 (z 0 w 1 w1 = w 1,b 1 = b 1, = l(y, z 2 z 2 z 2 z2 = z 2 v 2 v 2 z 1 z 1 z 1 = z 1, v 1 v 1 v1 =ṽ 1 w 1 Te first two expressions are familiar to us, and in fact we already calculated tem! We define and Ten, we can write δ 2 = δ 1 = z 2 l(y, z 2 (z 0 w1 = w 1,b 1 = b 1, z 1 l(y, z 2 (z 0 w1 = w 1,b 1 = b 1, δ 1 = δ 2 z 2 v 2 v 2 z 1 z 1 = z 1, Te last two expressions can be calculated te same way: z 1 v 1 = (ṽ 1 (1 (ṽ 1 = z 1 (1 z 1 v1 =ṽ 1 Te rigt expression is simply Summary v 1 = z 0 w 1 w 1 = w 1,b 1 = b 1 How to calculate te above efficiently? Note tat we needed te evaluation of all te functions/variables at te current values of te parameters. We do tis using a forward pass - Going forward in te grap, starting wit ṽ 1, using it to calculate z 1, ten z 2, and so fort. Ten, we calculate δ 2 as explained above, and use its value to calculate δ 1 - tis is te backward pass (or, backpropagation. It s easy to extend tis logic to any number of layers. Anoter way to tink about it is as a dynamic programming or memoization algoritm - we reuse te same expressions over and over, so we calculate tem only once, in te order needed to calculate tem. w1 = w 1,b 1 =
4 4 Lecture Many layers, many nodes We now generalize tis to several layers of several nodes. We do tis using matrix calculus - see also lesson scribe for a sligtly different derivation. In te general case, eac layer will now be a vector. Te parameters for te transition between layers are matrices and vectors: W L 2, b L 2 W L 1, b L 1 w L, b L... z L 2 z L 1 ŷ = z L Wit te notation v t+1 = W t+1 z t b t+1, we ave: W L 1, b L 1 w L, b L y... z L 2 v L 1 z L 1 v L z L l(y, z L We can now use matrix calculus to differentiate wit simplicity. (It s not new - it s stuff we ave all learned in multivariable calculus - Jacobians etc.. We omit te points were te derivatives are evaluated for clarity of presentation - te rationale is te same as before. Denote te i-t row of W t by w t,i. Ten, w t,i l(y, z L (z 0 = z L l(y, z L z L v L v L z L 1 z L 1 v L 1 v L 1 z L 2... Tis gradient is a row vector. We already know te first two expressions, z L l(y, z L and z L v L - tey are like before. Te expression v L z L 1 is a row vector - it is simply w L. Te expression z L 1 v L 1 is a matrix - te Jacobian z L 1 as a function of v L 1. It is simple to see its a diagonal matrix wit ((ṽ L 1 j on te diagonal j, j. Te matrix v L 1 z L 2 is simply W L 1! So we continue multiplying tese matrices, until te final matrix vt w t,i, wic is also easy to calculate - only te i-t row is nonzero, and it is z t. It s easy to verify tat tis gives exactly te same full algoritm as detailed in te lesson scribes. As before, we ave a forward pass to calculate all te v-s, and a backward pass to calculate all te δ-s, were δ t = z t l(y, z L (z 0. From tis, te calculation of te gradients are as described above and in te lesson. v t w t,i
5 8.2. DECISION TREES Decision Trees Terminology and Reminder Assume a binary classification setting (for every training sample, let f be te binary label. We like to decide in eac node on te split, i.e., te predicate to assign to te node. Te local parameters are q = Pr[f = 1], wic is te fraction of 1s in te examples reacing te node, u = Pr[ = 0] 1 is te fraction of samples for wic = 0 out of te samples reacing te node, p = Pr[f = 1 = 0] is te fraction of 1s in te samples reacing te node and aving = 0, and r = Pr[f = 1 = 1] is te fraction of 1s in te samples reacing te node and aving = 1. We ave tat q = up + (1 ur. (See Figure 8.1. Figure 8.1: Te split in a node Recall te decision-tree algoritm from class: We use a strictly convex node index function v( 2 tat associates a value to a node as a function of te proportion of positively labeled examples in te node (q using our above terminology. Now, by strict convexity of v( we ave v(q > u v(p + (1 uv(r And at a given node we seek to find a predicate tat splits in a way tat mostly reduces te rigt and side of te above inequality (te resulting node potential. 1 We use = 0 to indicate tat te predicate is false and = 1 for te case is true 2 An example of a split index is v(p = p log 2 p (1 p log 2 (1 p wic is te binary entropy function. (In class we normalized by multiplying by a alf, but tis will not make a difference.
6 6 Lecture Instability Example We consider te sample 2-feature binary labeled data in Figure 8.2a 3. Te root s optimal decision stump = x 1 < 0.6 reduces te potential 4 from te initial 1 (since te sample contains an equal number of positive and negative samples to v ( v ( (a Original sample set (b Sligt cange in one sample. Figure 8.2: Example of data for decision tree instability. Triangles are positively labeled and circles are negatively labeled We continue performing te splits and derive te decision tree of Figure 8.3. We can now consider wat will appen if we sligtly modify te location of a single point as follows. (See Figure 8.2b. Te modified data still as te root split = x 1 < 0.6 resulting in te same value 0.79, but for te root split = x 2 < 0.32 we ave 7 16 v ( v ( < 0.79 Tis implies tat te minor cange will cange te optimal predicate at te root and migt impact te entire tree. 3 Example from ttp:// sr ws0607/psr 0607 Cap10.pdf, slide 30 4 We use te entropy function trougout.
7 8.2. DECISION TREES 7 Figure 8.3: Te tree tat is built
LIMITATIONS OF EULER S METHOD FOR NUMERICAL INTEGRATION
LIMITATIONS OF EULER S METHOD FOR NUMERICAL INTEGRATION LAURA EVANS.. Introduction Not all differential equations can be explicitly solved for y. Tis can be problematic if we need to know te value of y
More informationFunction Composition and Chain Rules
Function Composition and s James K. Peterson Department of Biological Sciences and Department of Matematical Sciences Clemson University Marc 8, 2017 Outline 1 Function Composition and Continuity 2 Function
More informationSolutions to the Multivariable Calculus and Linear Algebra problems on the Comprehensive Examination of January 31, 2014
Solutions to te Multivariable Calculus and Linear Algebra problems on te Compreensive Examination of January 3, 24 Tere are 9 problems ( points eac, totaling 9 points) on tis portion of te examination.
More informationIntroduction to Derivatives
Introduction to Derivatives 5-Minute Review: Instantaneous Rates and Tangent Slope Recall te analogy tat we developed earlier First we saw tat te secant slope of te line troug te two points (a, f (a))
More information7.1 Using Antiderivatives to find Area
7.1 Using Antiderivatives to find Area Introduction finding te area under te grap of a nonnegative, continuous function f In tis section a formula is obtained for finding te area of te region bounded between
More informationDifferentiation in higher dimensions
Capter 2 Differentiation in iger dimensions 2.1 Te Total Derivative Recall tat if f : R R is a 1-variable function, and a R, we say tat f is differentiable at x = a if and only if te ratio f(a+) f(a) tends
More informationCalculus I Homework: The Derivative as a Function Page 1
Calculus I Homework: Te Derivative as a Function Page 1 Example (2.9.16) Make a careful sketc of te grap of f(x) = sin x and below it sketc te grap of f (x). Try to guess te formula of f (x) from its grap.
More informationMVT and Rolle s Theorem
AP Calculus CHAPTER 4 WORKSHEET APPLICATIONS OF DIFFERENTIATION MVT and Rolle s Teorem Name Seat # Date UNLESS INDICATED, DO NOT USE YOUR CALCULATOR FOR ANY OF THESE QUESTIONS In problems 1 and, state
More informationThe Derivative The rate of change
Calculus Lia Vas Te Derivative Te rate of cange Knowing and understanding te concept of derivative will enable you to answer te following questions. Let us consider a quantity wose size is described by
More informationHigher Derivatives. Differentiable Functions
Calculus 1 Lia Vas Higer Derivatives. Differentiable Functions Te second derivative. Te derivative itself can be considered as a function. Te instantaneous rate of cange of tis function is te second derivative.
More information1. Questions (a) through (e) refer to the graph of the function f given below. (A) 0 (B) 1 (C) 2 (D) 4 (E) does not exist
Mat 1120 Calculus Test 2. October 18, 2001 Your name Te multiple coice problems count 4 points eac. In te multiple coice section, circle te correct coice (or coices). You must sow your work on te oter
More information2.11 That s So Derivative
2.11 Tat s So Derivative Introduction to Differential Calculus Just as one defines instantaneous velocity in terms of average velocity, we now define te instantaneous rate of cange of a function at a point
More informationContinuity and Differentiability Worksheet
Continuity and Differentiability Workseet (Be sure tat you can also do te grapical eercises from te tet- Tese were not included below! Typical problems are like problems -3, p. 6; -3, p. 7; 33-34, p. 7;
More information4.2 - Richardson Extrapolation
. - Ricardson Extrapolation. Small-O Notation: Recall tat te big-o notation used to define te rate of convergence in Section.: Definition Let x n n converge to a number x. Suppose tat n n is a sequence
More informationLIMITS AND DERIVATIVES CONDITIONS FOR THE EXISTENCE OF A LIMIT
LIMITS AND DERIVATIVES Te limit of a function is defined as te value of y tat te curve approaces, as x approaces a particular value. Te limit of f (x) as x approaces a is written as f (x) approaces, as
More informationMath 312 Lecture Notes Modeling
Mat 3 Lecture Notes Modeling Warren Weckesser Department of Matematics Colgate University 5 7 January 006 Classifying Matematical Models An Example We consider te following scenario. During a storm, a
More informationBob Brown Math 251 Calculus 1 Chapter 3, Section 1 Completed 1 CCBC Dundalk
Bob Brown Mat 251 Calculus 1 Capter 3, Section 1 Completed 1 Te Tangent Line Problem Te idea of a tangent line first arises in geometry in te context of a circle. But before we jump into a discussion of
More informationSection 15.6 Directional Derivatives and the Gradient Vector
Section 15.6 Directional Derivatives and te Gradient Vector Finding rates of cange in different directions Recall tat wen we first started considering derivatives of functions of more tan one variable,
More information158 Calculus and Structures
58 Calculus and Structures CHAPTER PROPERTIES OF DERIVATIVES AND DIFFERENTIATION BY THE EASY WAY. Calculus and Structures 59 Copyrigt Capter PROPERTIES OF DERIVATIVES. INTRODUCTION In te last capter you
More informationMathematics 105 Calculus I. Exam 1. February 13, Solution Guide
Matematics 05 Calculus I Exam February, 009 Your Name: Solution Guide Tere are 6 total problems in tis exam. On eac problem, you must sow all your work, or oterwise torougly explain your conclusions. Tere
More information5.1 We will begin this section with the definition of a rational expression. We
Basic Properties and Reducing to Lowest Terms 5.1 We will begin tis section wit te definition of a rational epression. We will ten state te two basic properties associated wit rational epressions and go
More informationAverage Rate of Change
Te Derivative Tis can be tougt of as an attempt to draw a parallel (pysically and metaporically) between a line and a curve, applying te concept of slope to someting tat isn't actually straigt. Te slope
More informationSECTION 1.10: DIFFERENCE QUOTIENTS LEARNING OBJECTIVES
(Section.0: Difference Quotients).0. SECTION.0: DIFFERENCE QUOTIENTS LEARNING OBJECTIVES Define average rate of cange (and average velocity) algebraically and grapically. Be able to identify, construct,
More informationNatural Language Understanding. Recap: probability, language models, and feedforward networks. Lecture 12: Recurrent Neural Networks and LSTMs
Natural Language Understanding Lecture 12: Recurrent Neural Networks and LSTMs Recap: probability, language models, and feedforward networks Simple Recurrent Networks Adam Lopez Credits: Mirella Lapata
More information1 Limits and Continuity
1 Limits and Continuity 1.0 Tangent Lines, Velocities, Growt In tion 0.2, we estimated te slope of a line tangent to te grap of a function at a point. At te end of tion 0.3, we constructed a new function
More informationSin, Cos and All That
Sin, Cos and All Tat James K. Peterson Department of Biological Sciences and Department of Matematical Sciences Clemson University Marc 9, 2017 Outline Sin, Cos and all tat! A New Power Rule Derivatives
More information2.8 The Derivative as a Function
.8 Te Derivative as a Function Typically, we can find te derivative of a function f at many points of its domain: Definition. Suppose tat f is a function wic is differentiable at every point of an open
More informationAMS 147 Computational Methods and Applications Lecture 09 Copyright by Hongyun Wang, UCSC. Exact value. Effect of round-off error.
Lecture 09 Copyrigt by Hongyun Wang, UCSC Recap: Te total error in numerical differentiation fl( f ( x + fl( f ( x E T ( = f ( x Numerical result from a computer Exact value = e + f x+ Discretization error
More informationf a h f a h h lim lim
Te Derivative Te derivative of a function f at a (denoted f a) is f a if tis it exists. An alternative way of defining f a is f a x a fa fa fx fa x a Note tat te tangent line to te grap of f at te point
More informationNumerical Differentiation
Numerical Differentiation Finite Difference Formulas for te first derivative (Using Taylor Expansion tecnique) (section 8.3.) Suppose tat f() = g() is a function of te variable, and tat as 0 te function
More informationTHE IDEA OF DIFFERENTIABILITY FOR FUNCTIONS OF SEVERAL VARIABLES Math 225
THE IDEA OF DIFFERENTIABILITY FOR FUNCTIONS OF SEVERAL VARIABLES Mat 225 As we ave seen, te definition of derivative for a Mat 111 function g : R R and for acurveγ : R E n are te same, except for interpretation:
More informationDerivatives of Exponentials
mat 0 more on derivatives: day 0 Derivatives of Eponentials Recall tat DEFINITION... An eponential function as te form f () =a, were te base is a real number a > 0. Te domain of an eponential function
More information1 Calculus. 1.1 Gradients and the Derivative. Q f(x+h) f(x)
Calculus. Gradients and te Derivative Q f(x+) δy P T δx R f(x) 0 x x+ Let P (x, f(x)) and Q(x+, f(x+)) denote two points on te curve of te function y = f(x) and let R denote te point of intersection of
More informationCopyright c 2008 Kevin Long
Lecture 4 Numerical solution of initial value problems Te metods you ve learned so far ave obtained closed-form solutions to initial value problems. A closedform solution is an explicit algebriac formula
More informationHow to Find the Derivative of a Function: Calculus 1
Introduction How to Find te Derivative of a Function: Calculus 1 Calculus is not an easy matematics course Te fact tat you ave enrolled in suc a difficult subject indicates tat you are interested in te
More informationLab 6 Derivatives and Mutant Bacteria
Lab 6 Derivatives and Mutant Bacteria Date: September 27, 20 Assignment Due Date: October 4, 20 Goal: In tis lab you will furter explore te concept of a derivative using R. You will use your knowledge
More informationLecture XVII. Abstract We introduce the concept of directional derivative of a scalar function and discuss its relation with the gradient operator.
Lecture XVII Abstract We introduce te concept of directional derivative of a scalar function and discuss its relation wit te gradient operator. Directional derivative and gradient Te directional derivative
More informationFunction Composition and Chain Rules
Function Composition an Cain Rules James K. Peterson Department of Biological Sciences an Department of Matematical Sciences Clemson University November 2, 2018 Outline Function Composition an Continuity
More informationMain Points: 1. Limit of Difference Quotients. Prep 2.7: Derivatives and Rates of Change. Names of collaborators:
Name: Section: Names of collaborators: Main Points:. Definition of derivative as limit of difference quotients. Interpretation of derivative as slope of grap. Interpretation of derivative as instantaneous
More information2.3 Algebraic approach to limits
CHAPTER 2. LIMITS 32 2.3 Algebraic approac to its Now we start to learn ow to find its algebraically. Tis starts wit te simplest possible its, and ten builds tese up to more complicated examples. Fact.
More information1. Which one of the following expressions is not equal to all the others? 1 C. 1 D. 25x. 2. Simplify this expression as much as possible.
004 Algebra Pretest answers and scoring Part A. Multiple coice questions. Directions: Circle te letter ( A, B, C, D, or E ) net to te correct answer. points eac, no partial credit. Wic one of te following
More informationMAT244 - Ordinary Di erential Equations - Summer 2016 Assignment 2 Due: July 20, 2016
MAT244 - Ordinary Di erential Equations - Summer 206 Assignment 2 Due: July 20, 206 Full Name: Student #: Last First Indicate wic Tutorial Section you attend by filling in te appropriate circle: Tut 0
More information1 + t5 dt with respect to x. du = 2. dg du = f(u). du dx. dg dx = dg. du du. dg du. dx = 4x3. - page 1 -
Eercise. Find te derivative of g( 3 + t5 dt wit respect to. Solution: Te integrand is f(t + t 5. By FTC, f( + 5. Eercise. Find te derivative of e t2 dt wit respect to. Solution: Te integrand is f(t e t2.
More informationPrecalculus Test 2 Practice Questions Page 1. Note: You can expect other types of questions on the test than the ones presented here!
Precalculus Test 2 Practice Questions Page Note: You can expect oter types of questions on te test tan te ones presented ere! Questions Example. Find te vertex of te quadratic f(x) = 4x 2 x. Example 2.
More informationMinimizing D(Q,P) def = Q(h)
Inference Lecture 20: Variational Metods Kevin Murpy 29 November 2004 Inference means computing P( i v), were are te idden variables v are te visible variables. For discrete (eg binary) idden nodes, exact
More informationLesson 6: The Derivative
Lesson 6: Te Derivative Def. A difference quotient for a function as te form f(x + ) f(x) (x + ) x f(x + x) f(x) (x + x) x f(a + ) f(a) (a + ) a Notice tat a difference quotient always as te form of cange
More informationSection 3: The Derivative Definition of the Derivative
Capter 2 Te Derivative Business Calculus 85 Section 3: Te Derivative Definition of te Derivative Returning to te tangent slope problem from te first section, let's look at te problem of finding te slope
More information1 Lecture 13: The derivative as a function.
1 Lecture 13: Te erivative as a function. 1.1 Outline Definition of te erivative as a function. efinitions of ifferentiability. Power rule, erivative te exponential function Derivative of a sum an a multiple
More informationNUMERICAL DIFFERENTIATION. James T. Smith San Francisco State University. In calculus classes, you compute derivatives algebraically: for example,
NUMERICAL DIFFERENTIATION James T Smit San Francisco State University In calculus classes, you compute derivatives algebraically: for example, f( x) = x + x f ( x) = x x Tis tecnique requires your knowing
More information3.1 Extreme Values of a Function
.1 Etreme Values of a Function Section.1 Notes Page 1 One application of te derivative is finding minimum and maimum values off a grap. In precalculus we were only able to do tis wit quadratics by find
More informationCombining functions: algebraic methods
Combining functions: algebraic metods Functions can be added, subtracted, multiplied, divided, and raised to a power, just like numbers or algebra expressions. If f(x) = x 2 and g(x) = x + 2, clearly f(x)
More informationRecall from our discussion of continuity in lecture a function is continuous at a point x = a if and only if
Computational Aspects of its. Keeping te simple simple. Recall by elementary functions we mean :Polynomials (including linear and quadratic equations) Eponentials Logaritms Trig Functions Rational Functions
More informationExam 1 Review Solutions
Exam Review Solutions Please also review te old quizzes, and be sure tat you understand te omework problems. General notes: () Always give an algebraic reason for your answer (graps are not sufficient),
More informationTangent Lines-1. Tangent Lines
Tangent Lines- Tangent Lines In geometry, te tangent line to a circle wit centre O at a point A on te circle is defined to be te perpendicular line at A to te line OA. Te tangent lines ave te special property
More informationDifferential Calculus (The basics) Prepared by Mr. C. Hull
Differential Calculus Te basics) A : Limits In tis work on limits, we will deal only wit functions i.e. tose relationsips in wic an input variable ) defines a unique output variable y). Wen we work wit
More informationNotes on wavefunctions II: momentum wavefunctions
Notes on wavefunctions II: momentum wavefunctions and uncertainty Te state of a particle at any time is described by a wavefunction ψ(x). Tese wavefunction must cange wit time, since we know tat particles
More informationThe Derivative as a Function
Section 2.2 Te Derivative as a Function 200 Kiryl Tsiscanka Te Derivative as a Function DEFINITION: Te derivative of a function f at a number a, denoted by f (a), is if tis limit exists. f (a) f(a + )
More information4. The slope of the line 2x 7y = 8 is (a) 2/7 (b) 7/2 (c) 2 (d) 2/7 (e) None of these.
Mat 11. Test Form N Fall 016 Name. Instructions. Te first eleven problems are wort points eac. Te last six problems are wort 5 points eac. For te last six problems, you must use relevant metods of algebra
More information(4.2) -Richardson Extrapolation
(.) -Ricardson Extrapolation. Small-O Notation: Recall tat te big-o notation used to define te rate of convergence in Section.: Suppose tat lim G 0 and lim F L. Te function F is said to converge to L as
More information3.4 Worksheet: Proof of the Chain Rule NAME
Mat 1170 3.4 Workseet: Proof of te Cain Rule NAME Te Cain Rule So far we are able to differentiate all types of functions. For example: polynomials, rational, root, and trigonometric functions. We are
More informationA.P. CALCULUS (AB) Outline Chapter 3 (Derivatives)
A.P. CALCULUS (AB) Outline Capter 3 (Derivatives) NAME Date Previously in Capter 2 we determined te slope of a tangent line to a curve at a point as te limit of te slopes of secant lines using tat point
More informationChapter 5 FINITE DIFFERENCE METHOD (FDM)
MEE7 Computer Modeling Tecniques in Engineering Capter 5 FINITE DIFFERENCE METHOD (FDM) 5. Introduction to FDM Te finite difference tecniques are based upon approximations wic permit replacing differential
More informationMA455 Manifolds Solutions 1 May 2008
MA455 Manifolds Solutions 1 May 2008 1. (i) Given real numbers a < b, find a diffeomorpism (a, b) R. Solution: For example first map (a, b) to (0, π/2) and ten map (0, π/2) diffeomorpically to R using
More informationPolynomials 3: Powers of x 0 + h
near small binomial Capter 17 Polynomials 3: Powers of + Wile it is easy to compute wit powers of a counting-numerator, it is a lot more difficult to compute wit powers of a decimal-numerator. EXAMPLE
More information1 The concept of limits (p.217 p.229, p.242 p.249, p.255 p.256) 1.1 Limits Consider the function determined by the formula 3. x since at this point
MA00 Capter 6 Calculus and Basic Linear Algebra I Limits, Continuity and Differentiability Te concept of its (p.7 p.9, p.4 p.49, p.55 p.56). Limits Consider te function determined by te formula f Note
More informationTime (hours) Morphine sulfate (mg)
Mat Xa Fall 2002 Review Notes Limits and Definition of Derivative Important Information: 1 According to te most recent information from te Registrar, te Xa final exam will be eld from 9:15 am to 12:15
More informationMTH-112 Quiz 1 Name: # :
MTH- Quiz Name: # : Please write our name in te provided space. Simplif our answers. Sow our work.. Determine weter te given relation is a function. Give te domain and range of te relation.. Does te equation
More informationlecture 26: Richardson extrapolation
43 lecture 26: Ricardson extrapolation 35 Ricardson extrapolation, Romberg integration Trougout numerical analysis, one encounters procedures tat apply some simple approximation (eg, linear interpolation)
More informationContinuity. Example 1
Continuity MATH 1003 Calculus and Linear Algebra (Lecture 13.5) Maoseng Xiong Department of Matematics, HKUST A function f : (a, b) R is continuous at a point c (a, b) if 1. x c f (x) exists, 2. f (c)
More informationDynamics and Relativity
Dynamics and Relativity Stepen Siklos Lent term 2011 Hand-outs and examples seets, wic I will give out in lectures, are available from my web site www.damtp.cam.ac.uk/user/stcs/dynamics.tml Lecture notes,
More informationDifferentiation Rules c 2002 Donald Kreider and Dwight Lahr
Dierentiation Rules c 00 Donal Kreier an Dwigt Lar Te Power Rule is an example o a ierentiation rule. For unctions o te orm x r, were r is a constant real number, we can simply write own te erivative rater
More informationTHE IMPLICIT FUNCTION THEOREM
THE IMPLICIT FUNCTION THEOREM ALEXANDRU ALEMAN 1. Motivation and statement We want to understand a general situation wic occurs in almost any area wic uses matematics. Suppose we are given number of equations
More informationExponentials and Logarithms Review Part 2: Exponentials
Eponentials and Logaritms Review Part : Eponentials Notice te difference etween te functions: g( ) and f ( ) In te function g( ), te variale is te ase and te eponent is a constant. Tis is called a power
More information. If lim. x 2 x 1. f(x+h) f(x)
Review of Differential Calculus Wen te value of one variable y is uniquely determined by te value of anoter variable x, ten te relationsip between x and y is described by a function f tat assigns a value
More informationRightStart Mathematics
Most recent update: January 7, 2019 RigtStart Matematics Corrections and Updates for Level F/Grade 5 Lessons and Workseets, second edition LESSON / WORKSHEET CHANGE DATE CORRECTION OR UPDATE Lesson 7 04/18/2018
More informationMath 102 TEST CHAPTERS 3 & 4 Solutions & Comments Fall 2006
Mat 102 TEST CHAPTERS 3 & 4 Solutions & Comments Fall 2006 f(x+) f(x) 10 1. For f(x) = x 2 + 2x 5, find ))))))))) and simplify completely. NOTE: **f(x+) is NOT f(x)+! f(x+) f(x) (x+) 2 + 2(x+) 5 ( x 2
More information1 Solutions to the in class part
NAME: Solutions to te in class part. Te grap of a function f is given. Calculus wit Analytic Geometry I Exam, Friday, August 30, 0 SOLUTIONS (a) State te value of f(). (b) Estimate te value of f( ). (c)
More informationName: Answer Key No calculators. Show your work! 1. (21 points) All answers should either be,, a (finite) real number, or DNE ( does not exist ).
Mat - Final Exam August 3 rd, Name: Answer Key No calculators. Sow your work!. points) All answers sould eiter be,, a finite) real number, or DNE does not exist ). a) Use te grap of te function to evaluate
More informationConsider a function f we ll specify which assumptions we need to make about it in a minute. Let us reformulate the integral. 1 f(x) dx.
Capter 2 Integrals as sums and derivatives as differences We now switc to te simplest metods for integrating or differentiating a function from its function samples. A careful study of Taylor expansions
More informationACCESS TO SCIENCE, ENGINEERING AND AGRICULTURE: MATHEMATICS 1 MATH00030 SEMESTER /2019
ACCESS TO SCIENCE, ENGINEERING AND AGRICULTURE: MATHEMATICS MATH00030 SEMESTER 208/209 DR. ANTHONY BROWN 6. Differential Calculus 6.. Differentiation from First Principles. In tis capter, we will introduce
More informationGradient Descent etc.
1 Gradient Descent etc EE 13: Networked estimation and control Prof Kan) I DERIVATIVE Consider f : R R x fx) Te derivative is defined as d fx) = lim dx fx + ) fx) Te cain rule states tat if d d f gx) )
More information1 Power is transferred through a machine as shown. power input P I machine. power output P O. power loss P L. What is the efficiency of the machine?
1 1 Power is transferred troug a macine as sown. power input P I macine power output P O power loss P L Wat is te efficiency of te macine? P I P L P P P O + P L I O P L P O P I 2 ir in a bicycle pump is
More informationSECTION 3.2: DERIVATIVE FUNCTIONS and DIFFERENTIABILITY
(Section 3.2: Derivative Functions and Differentiability) 3.2.1 SECTION 3.2: DERIVATIVE FUNCTIONS and DIFFERENTIABILITY LEARNING OBJECTIVES Know, understand, and apply te Limit Definition of te Derivative
More informationTest 2 Review. 1. Find the determinant of the matrix below using (a) cofactor expansion and (b) row reduction. A = 3 2 =
Test Review Find te determinant of te matrix below using (a cofactor expansion and (b row reduction Answer: (a det + = (b Observe R R R R R R R R R Ten det B = (((det Hence det Use Cramer s rule to solve:
More informationMTH 119 Pre Calculus I Essex County College Division of Mathematics Sample Review Questions 1 Created April 17, 2007
MTH 9 Pre Calculus I Essex County College Division of Matematics Sample Review Questions Created April 7, 007 At Essex County College you sould be prepared to sow all work clearly and in order, ending
More informationMATH CALCULUS I 2.1: Derivatives and Rates of Change
MATH 12002 - CALCULUS I 2.1: Derivatives and Rates of Cange Professor Donald L. Wite Department of Matematical Sciences Kent State University D.L. Wite (Kent State University) 1 / 1 Introduction Our main
More informationThe Laplace equation, cylindrically or spherically symmetric case
Numerisce Metoden II, 7 4, und Übungen, 7 5 Course Notes, Summer Term 7 Some material and exercises Te Laplace equation, cylindrically or sperically symmetric case Electric and gravitational potential,
More informationDefinition of the Derivative
Te Limit Definition of te Derivative Tis Handout will: Define te limit grapically and algebraically Discuss, in detail, specific features of te definition of te derivative Provide a general strategy of
More informationSECTION 2.1 BASIC CALCULUS REVIEW
Tis capter covers just te very basics of wat you will nee moving forwar onto te subsequent capters. Tis is a summary capter, an will not cover te concepts in-ept. If you ve never seen calculus before,
More informationSolution. Solution. f (x) = (cos x)2 cos(2x) 2 sin(2x) 2 cos x ( sin x) (cos x) 4. f (π/4) = ( 2/2) ( 2/2) ( 2/2) ( 2/2) 4.
December 09, 20 Calculus PracticeTest s Name: (4 points) Find te absolute extrema of f(x) = x 3 0 on te interval [0, 4] Te derivative of f(x) is f (x) = 3x 2, wic is zero only at x = 0 Tus we only need
More informationExcerpt from "Calculus" 2013 AoPS Inc.
Excerpt from "Calculus" 03 AoPS Inc. Te term related rates refers to two quantities tat are dependent on eac oter and tat are canging over time. We can use te dependent relationsip between te quantities
More informationWYSE Academic Challenge 2004 Sectional Mathematics Solution Set
WYSE Academic Callenge 00 Sectional Matematics Solution Set. Answer: B. Since te equation can be written in te form x + y, we ave a major 5 semi-axis of lengt 5 and minor semi-axis of lengt. Tis means
More informationMA119-A Applied Calculus for Business Fall Homework 4 Solutions Due 9/29/ :30AM
MA9-A Applied Calculus for Business 006 Fall Homework Solutions Due 9/9/006 0:0AM. #0 Find te it 5 0 + +.. #8 Find te it. #6 Find te it 5 0 + + = (0) 5 0 (0) + (0) + =.!! r + +. r s r + + = () + 0 () +
More information1.5 Functions and Their Rates of Change
66_cpp-75.qd /6/8 4:8 PM Page 56 56 CHAPTER Introduction to Functions and Graps.5 Functions and Teir Rates of Cange Identif were a function is increasing or decreasing Use interval notation Use and interpret
More informationCSCE 478/878 Lecture 2: Concept Learning and the General-to-Specific Ordering
Outline Learning from eamples CSCE 78/878 Lecture : Concept Learning and te General-to-Specific Ordering Stepen D. Scott (Adapted from Tom Mitcell s slides) General-to-specific ordering over ypoteses Version
More informationHomework 1 Due: Wednesday, September 28, 2016
0-704 Information Processing and Learning Fall 06 Homework Due: Wednesday, September 8, 06 Notes: For positive integers k, [k] := {,..., k} denotes te set of te first k positive integers. Wen p and Y q
More informationRegularized Regression
Regularized Regression David M. Blei Columbia University December 5, 205 Modern regression problems are ig dimensional, wic means tat te number of covariates p is large. In practice statisticians regularize
More information. Compute the following limits.
Today: Tangent Lines and te Derivative at a Point Warmup:. Let f(x) =x. Compute te following limits. f( + ) f() (a) lim f( +) f( ) (b) lim. Let g(x) = x. Compute te following limits. g(3 + ) g(3) (a) lim
More informationLecture 21. Numerical differentiation. f ( x+h) f ( x) h h
Lecture Numerical differentiation Introduction We can analytically calculate te derivative of any elementary function, so tere migt seem to be no motivation for calculating derivatives numerically. However
More informationSection 2.4: Definition of Function
Section.4: Definition of Function Objectives Upon completion of tis lesson, you will be able to: Given a function, find and simplify a difference quotient: f ( + ) f ( ), 0 for: o Polynomial functions
More information