The simple essence of automatic differentiation
|
|
|
- Iris Lambert
- 5 years ago
- Views:
Transcription
1 The simple essence of automatic differentiation Conal Elliott Target January 2018 Conal Elliott January / 45
2 What s a derivative? Number Vector Covector Matrix Higher derivatives Chain rule for each. Conal Elliott The simple essence of automatic differentiation Jan / 45
3 What s a derivative? der :: pa Ñ bq Ñ pa Ñ pa bqq where f pa ` εq pf a ` der f a εq lim 0 εñ0 ε See Calculus on Manifolds by Michael Spivak. Conal Elliott The simple essence of automatic differentiation Jan / 45
4 Composition Sequential: p q :: pb Ñ cq Ñ pa Ñ bq Ñ pa Ñ cq pg f q a g pf aq der pg f q a der g pf aq der f a -- chain rule Parallel: pÿq :: pa Ñ cq Ñ pa Ñ dq Ñ pa Ñ c ˆ dq pf Ÿ gq a pf a, g aq der pf Ÿ gq a der f a Ÿ der g a Conal Elliott The simple essence of automatic differentiation Jan / 45
5 Compositionality Chain rule: der pg f q a der g pf aq der f a -- non-compositional To fix, combine regular result with derivative: andder :: pa Ñ bq Ñ pa Ñ pb ˆ pa bqqq andder f f Ÿ der f -- specification Often much work in common to f and der f. Conal Elliott The simple essence of automatic differentiation Jan / 45
6 Linear functions Linear functions are their own perfect linear approximations. der id a id der fst a fst der snd a snd... For linear functions f, i.e., andder f a pf a, f q andder f f Ÿ const f Conal Elliott The simple essence of automatic differentiation Jan / 45
7 Abstract algebra for functions class Category p q where id :: a a p q :: pb cq Ñ pa bq Ñ pa cq class Category p q ñ Cartesian p q where type a ˆp q b exl :: pa ˆp q bq a exr :: pa ˆp q bq b pÿq :: pa cq Ñ pa dq Ñ pa pc ˆp q dqq Plus laws and classes for arithmetic etc. Conal Elliott The simple essence of automatic differentiation Jan / 45
8 Simple automatic differentiation newtype D a b D pa Ñ b ˆ pa bqq lineard f D pf Ÿ const f q instance Category D where id lineard id D g D f D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g b u in pc, g 1 f 1 qq instance Cartesian D where exl lineard exl exr lineard exr D f Ÿ D g D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g a u in ppb, cq, f 1 Ÿ g 1 qq instance NumCat D where negate lineard negate add lineard add mul D pmul Ÿ λpa, bq Ñ pλpda, dbq Ñ b da ` a dbqq Conal Elliott The simple essence of automatic differentiation Jan / 45
9 Running examples sqr :: Num a ñ a Ñ a sqr a a a magsqr :: Num a ñ a ˆ a Ñ a magsqr pa, bq sqr a ` sqr b cossinprod :: Floating a ñ a ˆ a Ñ a ˆ a cossinprod px, yq pcos z, sin zq where z x y categorical vocabulary: sqr mul pid Ÿ idq magsqr add pmul pexl Ÿ exlq Ÿ mul pexr Ÿ exrqq cossinprod pcos Ÿ sinq mul Conal Elliott The simple essence of automatic differentiation Jan / 45
10 Visualizing computations magsqr pa, bq sqr a ` sqr b magsqr add pmul pexl Ÿ exlq Ÿ mul pexr Ÿ exrqq + Auto-generated from Haskell code. See Compiling to categories. Conal Elliott The simple essence of automatic differentiation Jan / 45
11 AD example sqr a a a sqr mul pid Ÿ idq + Conal Elliott The simple essence of automatic differentiation Jan / 45
12 AD example + magsqr pa, bq sqr a ` sqr b magsqr add pmul pexl Ÿ exlq Ÿ mul pexr Ÿ exrqq Conal Elliott The simple essence of automatic differentiation Jan / 45
13 AD example cos sin cossinprod px, yq pcos z, sin zq where z x y cossinprod pcos Ÿ sinq mul negate + sin cos Conal Elliott The simple essence of automatic differentiation Jan / 45
14 Generalizing AD newtype D a b D pa Ñ b ˆ pa bqq lineard f D pf Ÿ const f q instance Category D where id lineard id D g D f D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g b u in pc, g 1 f 1 qq instance Cartesian D where exl lineard exl exr lineard exr D f Ÿ D g D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g a u in ppb, cq, f 1 Ÿ g 1 qq Each D operation just uses corresponding p q operation. Generalize from p q to other cartesian categories. Conal Elliott The simple essence of automatic differentiation Jan / 45
15 Generalized AD newtype GAD p q a b D pa Ñ b ˆ pa bqq lineard f f 1 D pf Ÿ const f 1 q instance Category p q ñ Category pgad p qq where id lineard id id D g D f D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g b u in pc, g 1 f 1 qq instance Cartesian p q ñ Cartesian pgad p qq where exl lineard exl exl exr lineard exr exr D f Ÿ D g D pλa Ñ let tpb, f 1 q f a; pc, g 1 q g a u in ppb, cq, f 1 Ÿ g 1 qq instance... ñ NumCat D where negate lineard negate negate add lineard add add mul?? Conal Elliott The simple essence of automatic differentiation Jan / 45
16 Numeric operations Specific to (linear) functions: mul D pmul Ÿ λpa, bq Ñ pλpda, dbq Ñ b da ` a dbqq Rephrase: Now scale :: Multiplicative a ñ a Ñ pa aq scale u λv Ñ u v pźq :: Additive c ñ pa cq Ñ pb cq Ñ ppa ˆ bq cq f Ź g λpa, bq Ñ f a ` g b mul D pmul Ÿ pλpa, bq Ñ scale b Ź scale aqq Conal Elliott The simple essence of automatic differentiation Jan / 45
17 Linear arrow vocabulary class Category p q where id :: a a p q :: pb cq Ñ pa bq Ñ pa cq class Category p q ñ Cartesian p q where exl :: pa ˆ bq a exr :: pa ˆ bq b pÿq :: pa cq Ñ pa dq Ñ pa pc ˆ dqq class Category p q ñ Cocartesian p q where inl :: Additive b ñ a pa ˆ bq inr :: Additive a ñ b pa ˆ bq pźq :: Additive c ñ pa cq Ñ pb cq Ñ ppa ˆ bq cq class ScalarCat p q a where scale :: a Ñ pa aq Conal Elliott The simple essence of automatic differentiation Jan / 45
18 Linear transformations as functions newtype a Ñ`b AddFun pa Ñ bq instance Category pñ`q where type Ok pñ`q Additive id AddFun id p q innew 2 p q instance Cartesian pñ`q where exl AddFun exl exr AddFun exr pÿq innew 2 pÿq instance Cocartesian pñ`q where inl AddFun p, 0q inr AddFun p0, q pźq innew 2 pλf g px, yq Ñ f x ` g yq instance Multiplicative s ñ ScalarCat pñ`q s where scale s AddFun ps q Conal Elliott The simple essence of automatic differentiation Jan / 45
19 Extracting a data representation How to extract a matrix or gradient vector? Sample over a domain basis (rows of identity matrix). For n-dimensional domain, Make n passes. Each pass works on n-d sparse ( one-hot ) input. Very inefficient. For gradient-based optimization, High-dimensional domain. Very low-dimensional (1-D) codomain. Conal Elliott The simple essence of automatic differentiation Jan / 45
20 Generalized matrices newtype L s a b L pv s b pv s a sqq class HasV s a where type V s a :: Ñ tov :: a Ñ V s a s unv :: V s a s Ñ a -- Free vector space as representable functor instance Category pl sq where... instance Cartesian pl sq where... instance Cocartesian pl sq where... instance ScalarCat pl sq s where... Conal Elliott The simple essence of automatic differentiation Jan / 45
21 Core vocabulary Sufficient to build arbitrary matrices : scale :: a Ñ pa aq -- 1 ˆ 1 pźq :: pa cq Ñ pb cq Ñ ppa ˆ bq cq -- horizontal juxt pÿq :: pa cq Ñ pa dq Ñ pa pc ˆ dqq -- vertical juxt Types guarantee rectangularity. Conal Elliott The simple essence of automatic differentiation Jan / 45
22 Efficiency of composition Arrow composition is associative. Some associations are more efficient than others, so Associate optimally. Equivalent to matrix chain multiplication Opn log nq. Choice determined by types, i.e., compile-time information. All-right: forward mode AD (FAD). All-left: reverse mode AD (RAD). RAD is much better for gradient-based optimization. Conal Elliott The simple essence of automatic differentiation Jan / 45
23 Left-associating composition (RAD) CPS-like category: Represent a b by pb rq Ñ pa rq. Meaning: f ÞÑ p f q. Results in left-composition. itialize with id :: r r. Corresponds to a categorical pullback. Construct h der f a directly, without der f a. Conal Elliott The simple essence of automatic differentiation Jan / 45
24 instance Cocartesian p q ñ Cocartesian pcont p q r q where inl cont inl inr cont inr pźq innew 2 pλf g Ñ join pf Ÿ gqq Conal Elliott The simple essence of automatic differentiation Jan / 45 Continuation category newtype Cont p q r a b Cont ppb rq Ñ pa rqq cont :: Category p q ñ pa bq Ñ Cont p q r a b cont f Cont p f q -- Optimize uses instance Category pcont p q r q where type Ok pcont p q rq Ok p q id cont id p q innew 2 pflip p qq instance pcartesian p q, Cocartesian p qq ñ Cartesian pcont p q r q w exl cont exl exr cont exr pÿq innew 2 pλf g Ñ pf Ź gq unjoinq
25 Reverse-mode AD without tears type RAD s GAD pcont pl sqq Conal Elliott The simple essence of automatic differentiation Jan / 45
26 Left-associating composition (RAD) CPS-like category: Represent a b by pb rq Ñ pa rq. Meaning: f ÞÑ p f q. Results in left-composition. itialize with id :: r r. Corresponds to a categorical pullback. Construct h der f a directly, without der f a. We ve seen this trick before: Transforming naive reverse from quadratic to linear. Lists generalize to monoids, and monoids to categories. Conal Elliott The simple essence of automatic differentiation Jan / 45
27 One of my favorite papers Continuation-Based Program Transformation Strategies Mitch Wand, 1980, JACM. troduce a continuation argument, e.g., ra s Ñ ra s. Notice the continuations that arise, e.g., p ` asq. Find a data representation, e.g., as :: ra s Identify associative operation that represents composition, e.g., p `q, since p ` bsq p ` asq p ` pas ` bsqq. Conal Elliott The simple essence of automatic differentiation Jan / 45
28 Duality Vector space dual: u r, with u a vector space over r. If u has finite dimension, then u r u. For f :: u r, f dot v for some v :: u. Gradients are derivatives of functions with scalar codomain. Represent a b by pb rq Ñ pa rq by b Ñ a. Ideal for extracting gradient vector. Just apply to 1 (id). Conal Elliott The simple essence of automatic differentiation Jan / 45
29 Dual categories newtype Dual p q a b Dual pb aq instance Category p q ñ Category pdual p qq where id Dual id p q innew 2 pflip p qq instance Cocartesian p q ñ Cartesian pdual p qq where exl Dual inl exr Dual inr pÿq innew 2 pźq instance Cartesian p q ñ Cocartesian pdual p qq where inl Dual exl inr Dual exr pźq innew 2 pÿq instance ScalarCat p q s ñ ScalarCat pdual p qq s where scale s Dual pscale sq Conal Elliott The simple essence of automatic differentiation Jan / 45
30 Backpropagation type Backprop GAD pdual pñ`qq Conal Elliott The simple essence of automatic differentiation Jan / 45
31 RAD example (dual function) + + Conal Elliott The simple essence of automatic differentiation Jan / 45
32 RAD example (dual vector) Conal Elliott The simple essence of automatic differentiation Jan / 45
33 RAD example (dual function) + Conal Elliott The simple essence of automatic differentiation Jan / 45
34 RAD example (vector) 1.0 Conal Elliott The simple essence of automatic differentiation Jan / 45
35 RAD example (dual function) 0.0 Conal Elliott The simple essence of automatic differentiation Jan / 45
36 RAD example (dual vector) Conal Elliott The simple essence of automatic differentiation Jan / 45
37 RAD example (dual function) + Conal Elliott The simple essence of automatic differentiation Jan / 45
38 RAD example (dual vector) + Conal Elliott The simple essence of automatic differentiation Jan / 45
39 RAD example (dual function) Conal Elliott The simple essence of automatic differentiation Jan / 45
40 RAD example (dual vector) Conal Elliott The simple essence of automatic differentiation Jan / 45
41 RAD example (dual function) cos sin cos sin cos sin Conal Elliott The simple essence of automatic differentiation Jan / 45
42 RAD example (matrix) cos sin cos sin negate negate Conal Elliott The simple essence of automatic differentiation Jan / 45
43 cremental evaluation + if if + if if if if Conal Elliott The simple essence of automatic differentiation Jan / 45
44 Symbolic vs automatic differentiation Often described as opposing techniques: Symbolic: Apply differentiation rules symbolically. Can duplicate much work. Needs algebraic manipulation. Automatic: FAD: easy to implement but often inefficient. RAD: efficient but tricky to implement. Another view: AD is SD done by a compiler. Compilers already work symbolically and preserve sharing. Conal Elliott The simple essence of automatic differentiation Jan / 45
45 Conclusions Simple AD algorithm, specializing to forward, reverse, mixed. No graphs; no partial derivatives. One rule per combining form: p q, pÿq, pźq. Reverse mode via simple, general dual construction. Generalizes to derivative categories other than linear maps. Differentiate regular Haskell code (via plugin). Conal Elliott The simple essence of automatic differentiation Jan / 45
The simple essence of automatic differentiation
The simple essence of automatic differentiation Conal Elliott Target January/June 2018 Conal Elliott January/June 2018 1 / 51 Differentiable programming made easy Current AI revolution runs on large data,
The simple essence of automatic differentiation
The simple essence of automatic differentiation Conal Elliott Target January/June 2018 Conal Elliott January/June 2018 1 / 52 Machine learning: promise and problems Current AI revolution runs on large
The essence and origins of FRP
The essence and origins of FRP Conal Elliott June 13, 2015 Conal Elliott The essence and origins of FRP June 13, 2015 1 / 31 What is FRP? Conal Elliott The essence and origins of FRP June 13, 2015 2 /
Transformation Matrices; Geometric and Otherwise As examples, consider the transformation matrices of the C 3v
Transformation Matrices; Geometric and Otherwise As examples, consider the transformation matrices of the v group. The form of these matrices depends on the basis we choose. Examples: Cartesian vectors:
NOTES WEEK 15 DAY 1 SCOT ADAMS
NOTES WEEK 15 DAY 1 SCOT ADAMS We fix some notation for the entire class today: Let n P N, W : R n, : 2 P N pw q, W : LpW, W q, I : id W P W, z : 0 W 0 n. Note that W LpR n, R n q. Recall, for all T P
1. Examples. We did most of the following in class in passing. Now compile all that data.
SOLUTIONS Math A4900 Homework 12 11/22/2017 1. Examples. We did most of the following in class in passing. Now compile all that data. (a) Favorite examples: Let R tr, Z, Z{3Z, Z{6Z, M 2 prq, Rrxs, Zrxs,
String diagrams. a functorial semantics of proofs and programs. Paul-André Melliès. CNRS, Université Paris Denis Diderot.
String diagrams a functorial semantics of proofs and programs Paul-André Melliès CNRS, Université Paris Denis Diderot Hopf-in-Lux Luxembourg 17 July 2009 1 Proof-knots Aim: formulate an algebra of these
Structures in tangent categories
Structures in tangent categories Geoff Cruttwell Mount Allison University (joint work with Robin Cockett) Category Theory 2014 Cambridge, UK, June 30th, 2014 Outline What are tangent categories? Definitions:
RELATIVE THEORY IN SUBCATEGORIES. Introduction
RELATIVE THEORY IN SUBCATEGORIES SOUD KHALIFA MOHAMMED Abstract. We generalize the relative (co)tilting theory of Auslander- Solberg [9, 1] in the category mod Λ of finitely generated left modules over
Whitehead problems for words in Z m ˆ F n
Whitehead problems for words in Z m ˆ F n Jordi Delgado Dept. Mat. Apl. III Universitat Politècnica de Catalunya, Manresa, Barcelona jorge.delgado@upc.edu jdelgado.upc@gmail.com January 15, 2013 Abstract
4 Relativistic kinematics
4 Relativistic kinematics In astrophysics, we are often dealing with relativistic particles that are being accelerated by electric or magnetic forces. This produces radiation, typically in the form of
Categorical Databases
Categorical Databases Patrick Schultz, David Spivak MIT Ryan Wisnesky Categorical Informatics and others October 2017 Introduction This talk describes a new algebraic (purely equational) way to formalize
L E C T U R E 2 1 : M AT R I X T R A N S F O R M AT I O N S. Monday, November 14
L E C T U R E 2 1 : M AT R I X T R A N S F O R M AT I O N S Monday, November 14 In this lecture we want to consider functions between vector spaces. Recall that a function is simply a rule that takes an
L11: Algebraic Path Problems with applications to Internet Routing Lecture 15. Path Weight with functions on arcs?
L11: Algebraic Path Problems with applications to Internet Routing Lecture 15 Timothy G. Griffin timothy.griffin@cl.cam.ac.uk Computer Laboratory University of Cambridge, UK Michaelmas Term, 2016 tgg22
Linear Algebra Basics
Linear Algebra Basics For the next chapter, understanding matrices and how to do computations with them will be crucial. So, a good first place to start is perhaps What is a matrix? A matrix A is an array
Beautiful Differentiation
Under consideration for publication in J. Functional Programming 1 Beautiful Differentiation Conal M. Elliott LambdaPix (e-mail: conal@conal.net) Abstract Automatic differentiation (AD) is a precise, efficient,
t-deformations of Grothendieck rings as quantum cluster algebras
as quantum cluster algebras Universite Paris-Diderot June 7, 2018 Motivation U q pĝq : untwisted quantum Kac-Moody affine algebra of simply laced type, where q P C is not a root of unity, C : the category
MTH 505: Number Theory Spring 2017
MTH 505: Number Theory Spring 017 Homework 4 Drew Armstrong 4.1. (Squares Mod 4). We say that an element ras n P Z{nZ is square if there exists an element rxs n P Z{nZ such that ras n prxs n q rx s n.
COMPLEX NUMBERS LAURENT DEMONET
COMPLEX NUMBERS LAURENT DEMONET Contents Introduction: numbers 1 1. Construction of complex numbers. Conjugation 4 3. Geometric representation of complex numbers 5 3.1. The unit circle 6 3.. Modulus of
PRODUCT MEASURES AND FUBINI S THEOREM
CHAPTER 6 PRODUCT MEASURES AND FUBINI S THEOREM All students learn in elementary calculus to evaluate a double integral by iteration. The theorem justifying this process is called Fubini s theorem. However,
Ground States are generically a periodic orbit
Gonzalo Contreras CIMAT Guanajuato, Mexico Ergodic Optimization and related topics USP, Sao Paulo December 11, 2013 Expanding map X compact metric space. T : X Ñ X an expanding map i.e. T P C 0, Dd P Z`,
arxiv: v1 [math.ra] 24 Jun 2013
![arxiv: v1 [math.ra] 24 Jun 2013](/thumbs/76/73368948.jpg "arxiv: v1 [math.ra] 24 Jun 2013")
DISTRIBUTIVITY IN SKEW LATTICES MICHAEL KINYON, JONATHAN LEECH, AND JOÃO PITA COSTA arxiv:1306.5598v1 [math.ra] 24 Jun 2013 Abstract. Distributive skew lattices satisfying x ^ py _ zq ^ x px ^ y ^ xq _
8-1: Backpropagation Prof. J.C. Kao, UCLA. Backpropagation. Chain rule for the derivatives Backpropagation graphs Examples
8-1: Backpropagation Prof. J.C. Kao, UCLA Backpropagation Chain rule for the derivatives Backpropagation graphs Examples 8-2: Backpropagation Prof. J.C. Kao, UCLA Motivation for backpropagation To do gradient
DS-GA 3001: PROBLEM SESSION 2 SOLUTIONS VLADIMIR KOBZAR
DS-GA 3: PROBLEM SESSION SOLUTIONS VLADIMIR KOBZAR Note: In this discussion we allude to the fact the dimension of a vector space is given by the number of basis vectors. We will shortly establish this
Exact solutions of (0,2) Landau-Ginzburg models
Exact solutions of (0,2) Landau-Ginzburg models 1608.07753 w/ A. Gadde Pavel Putrov IAS, Princeton November 28, 2016 1 / 17 Motivaiton Known: N p0, 0q: real boson φ, V pφq φ 2n IR ÝÑ n-th unitary minimal
Chapter 2: Vector Geometry
Chapter 2: Vector Geometry Daniel Chan UNSW Semester 1 2018 Daniel Chan (UNSW) Chapter 2: Vector Geometry Semester 1 2018 1 / 32 Goals of this chapter In this chapter, we will answer the following geometric
MATH 360 Final Exam Thursday, December 14, a n 2. a n 1 1
MATH 36 Final Exam Thursday, December 4, 27 Name. The sequence ta n u is defined by a and a n (a) Prove that lim a n exists by showing that ta n u is bounded and monotonic and invoking an appropriate result.
Rank-one LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the well-known Lyapunov's matrix inequality about
Rank-one LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the well-known Lyapunov's matrix inequality about the location of the eigenvalues of a matrix
NOTES WEEK 02 DAY 1. THEOREM 0.3. Let A, B and C be sets. Then
NOTES WEEK 02 DAY 1 SCOT ADAMS LEMMA 0.1. @propositions P, Q, R, rp or pq&rqs rp p or Qq&pP or Rqs. THEOREM 0.2. Let A and B be sets. Then (i) A X B B X A, and (ii) A Y B B Y A THEOREM 0.3. Let A, B and
A Full RNS Implementation of Fan and Vercauteren Somewhat Homomorphic Encryption Scheme
A Full RNS Implementation of Fan and Vercauteren Somewhat Homomorphic Encryption Scheme Presented by: Vincent Zucca 1 Joint work with: Jean-Claude Bajard 1, Julien Eynard 2 and Anwar Hasan 2 1 Sorbonne
3. Signatures Problem 27. Show that if K` and K differ by a crossing change, then σpk`q
1. Introduction Problem 1. Prove that H 1 ps 3 zk; Zq Z and H 2 ps 3 zk; Zq 0 without using the Alexander duality. Problem 2. Compute the knot group of the trefoil. Show that it is not trivial. Problem
NOTES WEEK 13 DAY 2 SCOT ADAMS
NOTES WEEK 13 DAY 2 SCOT ADAMS Recall: Let px, dq be a metric space. Then, for all S Ď X, we have p S is sequentially compact q ñ p S is closed and bounded q. DEFINITION 0.1. Let px, dq be a metric space.
Entropy and Ergodic Theory Notes 22: The Kolmogorov Sinai entropy of a measure-preserving system
Entropy and Ergodic Theory Notes 22: The Kolmogorov Sinai entropy of a measure-preserving system 1 Joinings and channels 1.1 Joinings Definition 1. If px, µ, T q and py, ν, Sq are MPSs, then a joining
Traces of rationality of Darmon points
Traces of rationality of Darmon points [BD09] Bertolini Darmon, The rationality of Stark Heegner points over genus fields of real quadratic fields Seminari de Teoria de Nombres de Barcelona Barcelona,
APPROXIMATE HOMOMORPHISMS BETWEEN THE BOOLEAN CUBE AND GROUPS OF PRIME ORDER
APPROXIMATE HOMOMORPHISMS BETWEEN THE BOOLEAN CUBE AND GROUPS OF PRIME ORDER TOM SANDERS The purpose of this note is to highlight a question raised by Shachar Lovett [Lov], and to offer some motivation
Approximation Algorithms using Allegories and Coq
Approximation Algorithms using Allegories and Coq Durjay Chowdhury Supervisor: Dr. Michael Winter Submitted in partial fulfilment of the requirements for the degree of Master of Science Department of Computer
Diffeological Spaces and Denotational Semantics for Differential Programming
Kammar, Staton, andvákár Diffeological Spaces and Denotational Semantics for Differential Programming Ohad Kammar, Sam Staton, and Matthijs Vákár MFPS 2018 Halifax 8 June 2018 What is differential programming?
Cohomology: A Mirror of Homotopy
Cohomology: A Mirror of Homotopy Agnès Beaudry University of Chicago September 19, 1 Spectra Definition Top is the category of based topological spaces with based continuous functions rx, Y s denotes the
MATH 101B: ALGEBRA II PART A: HOMOLOGICAL ALGEBRA 23
MATH 101B: ALGEBRA II PART A: HOMOLOGICAL ALGEBRA 23 6.4. Homotopy uniqueness of projective resolutions. Here I proved that the projective resolution of any R-module (or any object of an abelian category
NOTES WEEK 10 DAY 2. Unassigned HW: Let V and W be finite dimensional vector spaces and let x P V. Show, for all f, g : V W, that
NOTES WEEK 10 DAY 2 SCOT ADAMS Unassigned HW: Let V and W be finite dimensional vector spaces and let x P V. Show, for all f, g : V W, that D x pf ` gq pd x fq ` pd x gq. Also, show, for all c P R, for
Exotic Crossed Products
Exotic Crossed Products Rufus Willett (joint with Alcides Buss and Siegfried Echterhoff, and with Paul Baum and Erik Guentner) University of Hawai i WCOAS, University of Denver, November 2014 1 / 15 G
Existence of weak adiabatic limit in almost all models of perturbative QFT
Existence of weak adiabatic limit in almost all models of perturbative QFT Paweł Duch Jagiellonian University, Cracow, Poland LQP 40 Foundations and Constructive Aspects of Quantum Field Theory, 23.06.2017
b. Induction (LfP 50 3)
b. Induction (LfP 50 3) b.i. The principle of mathematical induction The principle of mathematical induction is a property of natural numbers. b.i.1. Natural numbers The natural numbers are the non-negative
4.3 Equations in 3-space
4.3 Equations in 3-space istance can be used to define functions from a 3-space R 3 to the line R. Let P be a fixed point in the 3-space R 3 (say, with coordinates P (2, 5, 7)). Consider a function f :
Teooriaseminar. TTÜ Küberneetika Instituut. May 10, Categorical Models. for Two Intuitionistic Modal Logics. Wolfgang Jeltsch.
TTÜ Küberneetika Instituut Teooriaseminar May 10, 2012 1 2 3 4 1 2 3 4 Modal logics used to deal with things like possibility, belief, and time in this talk only time two new operators and : ϕ now and
Very quick introduction to the conformal group and cft
CHAPTER 1 Very quick introduction to the conformal group and cft The world of Conformal field theory is big and, like many theories in physics, it can be studied in many ways which may seem very confusing
MATH 260 Class notes/questions January 10, 2013
MATH 26 Class notes/questions January, 2 Linear transformations Last semester, you studied vector spaces (linear spaces) their bases, dimension, the ideas of linear dependence and linear independence Now
Sobolev-Type Extremal Polynomials
Soolev-Type Extremal Polyomials (case of the sigular measures) Ael Díaz-Gozález Joi with Héctor Pijeira-Carera ULL 2016 Soolev-Type Extremal Polyomials adiazgo@math.uc3m.es 1 / 18 Moic extremal polyomials
p,q H (X), H (Y ) ), where the index p has the same meaning as the
There are two Eilenberg-Moore spectral sequences that we shall consider, one for homology and the other for cohomology. In contrast with the situation for the Serre spectral sequence, for the Eilenberg-Moore
NOTES WEEK 01 DAY 1 SCOT ADAMS
NOTES WEEK 01 DAY 1 SCOT ADAMS Question: What is Mathematics? Answer: The study of absolute truth. Question: Why is it so hard to teach and to learn? Answer: One must learn to play a variety of games called
J. D. H. Smith ONE-SIDED QUANTUM QUASIGROUPS AND LOOPS
DEMONSTRTIO MTHEMTIC Vol. XLVIII No 4 2015 J. D. H. Smith ONE-SIDED QUNTUM QUSIGROUPS ND LOOPS Communicated by. Romanowska bstract. Quantum quasigroups and quantum loops are self-dual objects providing
Hybrid Systems - Lecture n. 3 Lyapunov stability
OUTLINE Focus: stability of equilibrium point Hybrid Systems - Lecture n. 3 Lyapunov stability Maria Prandini DEI - Politecnico di Milano E-mail: prandini@elet.polimi.it continuous systems decribed by
Vectors. January 13, 2013
Vectors January 13, 2013 The simplest tensors are scalars, which are the measurable quantities of a theory, left invariant by symmetry transformations. By far the most common non-scalars are the vectors,
Homework for MATH 4604 (Advanced Calculus II) Spring 2017
Homework for MATH 4604 (Advanced Calculus II) Spring 2017 Homework 14: Due on Tuesday 2 May 55. Let m, n P N, A P R mˆn and v P R n. Show: L A pvq 2 ď A 2 v 2. 56. Let n P N and let A P R nˆn. Let I n
ADVANCE TOPICS IN ANALYSIS - REAL. 8 September September 2011
ADVANCE TOPICS IN ANALYSIS - REAL NOTES COMPILED BY KATO LA Introductions 8 September 011 15 September 011 Nested Interval Theorem: If A 1 ra 1, b 1 s, A ra, b s,, A n ra n, b n s, and A 1 Ě A Ě Ě A n
Approximation in the Zygmund Class
Approximation in the Zygmund Class Odí Soler i Gibert Joint work with Artur Nicolau Universitat Autònoma de Barcelona New Developments in Complex Analysis and Function Theory, 02 July 2018 Approximation
Exam 1 Solutions. Solution: The 16 contributes 5 to the total and contributes 2. All totaled, there are 5 ˆ 2 10 abelian groups.
Math 5372 Spring 2014 Exam 1 Solutions 1. (15 points) How many abelian groups are there of order: (a) 16 For any prime p, there are as many groups of order p k as there are partitions of k. The number
Mariusz Jurkiewicz, Bogdan Przeradzki EXISTENCE OF SOLUTIONS FOR HIGHER ORDER BVP WITH PARAMETERS VIA CRITICAL POINT THEORY
DEMONSTRATIO MATHEMATICA Vol. XLVIII No 1 215 Mariusz Jurkiewicz, Bogdan Przeradzki EXISTENCE OF SOLUTIONS FOR HIGHER ORDER BVP WITH PARAMETERS VIA CRITICAL POINT THEORY Communicated by E. Zadrzyńska Abstract.
Review of Coordinate Systems
Vector in 2 R and 3 R Review of Coordinate Systems Used to describe the position of a point in space Common coordinate systems are: Cartesian Polar Cartesian Coordinate System Also called rectangular coordinate
NOTES WEEK 14 DAY 2 SCOT ADAMS
NOTES WEEK 14 DAY 2 SCOT ADAMS We igligt tat it s possible to ave two topological spaces and a continuous bijection from te one to te oter wose inverse is not continuous: Let I : r0, 2πq and let C : tpx,
1.1.1 Algebraic Operations
1.1.1 Algebraic Operations We need to learn how our basic algebraic operations interact. When confronted with many operations, we follow the order of operations: Parentheses Exponentials Multiplication
1 Differentiable manifolds and smooth maps
1 Differentiable manifolds and smooth maps Last updated: April 14, 2011. 1.1 Examples and definitions Roughly, manifolds are sets where one can introduce coordinates. An n-dimensional manifold is a set
Relational Algebra by Way of Adjunctions. Jeremy Gibbons (joint work with Fritz Henglein, Ralf Hinze, Nicolas Wu) DBPL, October 2015
Relational Algebra by Way of Adjunctions Jeremy Gibbons (joint work with Fritz Henglein, Ralf Hinze, Nicolas Wu) DBPL, October 2015 Relational Algebra by Way of Adjunctions 2 1. Summary bulk types (sets,
Singular integral operators and the Riesz transform
Singular integral operators and the Riesz transform Jordan Bell jordan.bell@gmail.com Department of Mathematics, University of Toronto November 17, 017 1 Calderón-Zygmund kernels Let ω n 1 be the measure
Intro Vectors 2D implicit curves 2D parametric curves. Graphics 2011/2012, 4th quarter. Lecture 2: vectors, curves, and surfaces
Lecture 2, curves, and surfaces Organizational remarks Tutorials: Tutorial 1 will be online later today TA sessions for questions start next week Practicals: Exams: Make sure to find a team partner very
Ohio s Learning Standards-Extended. Mathematics. The Real Number System Complexity a Complexity b Complexity c
Ohio s Learning Standards-Extended Mathematics The Real Number System Complexity a Complexity b Complexity c Extend the properties of exponents to rational exponents N.RN.1 Explain how the definition of
MTH 505: Number Theory Spring 2017
MTH 505: Number Theory Spring 2017 Homework 1 Drew Armstrong 1.1. From pn, σ, 0q to pn, `,, 0, 1q. Recall Peano s four axioms for the natural numbers: (P1) There exists a special element called 0 P N.
AanumntBAasciAs. l e t e s auas trasuarbe, amtima*. pay Bna. aaeh t!iacttign. Xat as eling te Trndi'aBd^glit!
- [ - --- --- ~ - 5 4 G 4? G 8 0 0 0 7 0 - Q - - - 6 8 7 2 75 00 - [ 7-6 - - Q - ] z - 9 - G - 0 - - z / - ] G / - - 4-6 7 - z - 6 - - z - - - - - - G z / - - - G 0 Zz 4 z4 5? - - Z z 2 - - {- 9 9? Z G
NOTES WEEK 04 DAY 1 SCOT ADAMS
NOTES WEEK 0 DAY 1 SCOT ADAMS DEFINITION 01 Let m, n P N, B P BpR m, R n q Let e 1,, e m be the standard basis of R m Let f 1,, f n be the standard basis of R n Then we define rbs P R nˆm by rbs ji Bpe
CATEGORY THEORY. Cats have been around for 70 years. Eilenberg + Mac Lane =. Cats are about building bridges between different parts of maths.
CATEGORY THEORY PROFESSOR PETER JOHNSTONE Cats have been around for 70 years. Eilenberg + Mac Lane =. Cats are about building bridges between different parts of maths. Definition 1.1. A category C consists
Introduction to Applied Algebraic Topology
Introduction to Applied Algebraic Topology Tom Needham Last Updated: December 10, 2017 These are lecture notes for the course MATH 4570 at the Ohio State University. They are a work in progress and certainly
Diffeological Spaces and Denotational Semantics for Differential Programming
Diffeological Spaces and Denotational Semantics for Differential Programming Ohad Kammar, Sam Staton, and Matthijs Vákár Domains 2018 Oxford 8 July 2018 What is differential programming? PL in which all
PROBABILITY 2018 HANDOUT 2: GAUSSIAN DISTRIBUTION
PROAILITY 218 HANDOUT 2: GAUSSIAN DISTRIUTION GIOVANNI PISTONE Contents 1. Standard Gaussian Distribution 1 2. Positive Definite Matrices 5 3. General Gaussian Distribution 6 4. Independence of Jointly
Topological Groupoids and Exponentiability
Topological Groupoids and Exponentiability Susan Niefield (joint with Dorette Pronk) July 2017 Overview Goal: Study exponentiability in categories of topological groupoid. Starting Point: Consider exponentiability
JUSTIFYING BOOLE S ALGEBRA of LOGIC for CLASSES
Outline The Paradigm The Starting Point Boole s Partial Algebras Boole s Translations Laws and Rules of Inference Characteristic Functions Signed Multisets NCR 1 R01 Horn Formulas Relativizing Quantifiers
Review++ of linear algebra continues
Review++ of linear algebra continues Recall a matrix A P M m,n pf q is an array 1 1 2 1 1 n A p j i q 1 2 2 2 2 n...... 1 m n m This encodes a map ' P HompF n,f m q via the images of basis vectors of F
K. P. R. Rao, K. R. K. Rao UNIQUE COMMON FIXED POINT THEOREMS FOR PAIRS OF HYBRID MAPS UNDER A NEW CONDITION IN PARTIAL METRIC SPACES
DEMONSTRATIO MATHEMATICA Vol. XLVII No 3 204 K. P. R. Rao, K. R. K. Rao UNIQUE COMMON FIXED POINT THEOREMS FOR PAIRS OF HYBRID MAPS UNDER A NEW CONDITION IN PARTIAL METRIC SPACES Abstract. In this paper,
Vectors. Introduction. Prof Dr Ahmet ATAÇ
Chapter 3 Vectors Vectors Vector quantities Physical quantities that have both n u m e r i c a l a n d d i r e c t i o n a l properties Mathematical operations of vectors in this chapter A d d i t i o
C*-algebras - a case study
- a case study Definition Suppose that H is a Hilbert space. A C -algebra is an operator-norm closed -subalgebra of BpHq. are closed under ultraproducts and subalgebras so they should be captured by continous
A Practical Randomized CP Tensor Decomposition
A Practical Randomized CP Tensor Decomposition Casey Battaglino, Grey Ballard 2, and Tamara G. Kolda 3 SIAM AN 207, Pittsburgh, PA Georgia Tech Computational Sci. and Engr. 2 Wake Forest University 3 Sandia
CBSE 2018 ANNUAL EXAMINATION DELHI
CBSE 08 ANNUAL EXAMINATION DELHI (Series SGN Code No 65/ : Delhi Region) Ma Marks : 00 Time Allowed : Hours SECTION A Q0 Find the value of tan cot ( ) Sol 5 5 tan cot ( ) tan tan cot cot 6 6 6 0 a Q0 If
Dr. Marques Sophie Algebra 1 Spring Semester 2017 Problem Set 9
Dr. Marques Sophie Algebra Spring Semester 207 Office 59 marques@cims.nyu.edu Problem Set 9 Exercise 0 : Prove that every group of order G 28 must contain a normal subgroup of order 7. (Why should it contain
The Algebra of Tensors; Tensors on a Vector Space Definition. Suppose V 1,,V k and W are vector spaces. A map. F : V 1 V k
The Algebra of Tensors; Tensors on a Vector Space Definition. Suppose V 1,,V k and W are vector spaces. A map F : V 1 V k is said to be multilinear if it is linear as a function of each variable seperately:
Entropy and Ergodic Theory Lecture 28: Sinai s and Ornstein s theorems, II
Entropy and Ergodic Theory Lecture 28: Sinai s and Ornstein s theorems, II 1 Proof of the update proposition Consider again an ergodic and atomless source ra Z, µs and a Bernoulli source rb Z, pˆz s such
arxiv: v2 [math.pr] 26 Jun 2017
![arxiv: v2 [math.pr] 26 Jun 2017](/thumbs/83/87237169.jpg "arxiv: v2 [math.pr] 26 Jun 2017")
A COUPLE OF REMARKS ON THE CONVERGENCE OF σ-fields ON PROBABILITY SPACES MATIJA VIDMAR arxiv:06.02848v2 [math.pr] 26 Jun 2017 Abstract. The following modes of convergence of sub-σ-fields on a given probability
Hybrid Systems Course Lyapunov stability
Hybrid Systems Course Lyapunov stability OUTLINE Focus: stability of an equilibrium point continuous systems decribed by ordinary differential equations (brief review) hybrid automata OUTLINE Focus: stability
Numerical Analysis Fall. Gauss Elimination
Numerical Analysis 2015 Fall Gauss Elimination Solving systems m g g m m g x x x k k k k k k k k k 3 2 1 3 2 1 3 3 3 2 3 2 2 2 1 0 0 Graphical Method For small sets of simultaneous equations, graphing
Flag Varieties. Matthew Goroff November 2, 2016
Flag Varieties Matthew Goroff November 2, 2016 1. Grassmannian Variety Definition 1.1: Let V be a k-vector space of dimension n. The Grassmannian Grpr, V q is the set of r-dimensional subspaces of V. It
CSC321 Lecture 6: Backpropagation
CSC321 Lecture 6: Backpropagation Roger Grosse Roger Grosse CSC321 Lecture 6: Backpropagation 1 / 21 Overview We ve seen that multilayer neural networks are powerful. But how can we actually learn them?
REAL ANALYSIS II TAKE HOME EXAM. T. Tao s Lecture Notes Set 5
REAL ANALYSIS II TAKE HOME EXAM CİHAN BAHRAN T. Tao s Lecture Notes Set 5 1. Suppose that te 1, e 2, e 3,... u is a countable orthonormal system in a complex Hilbert space H, and c 1, c 2,... is a sequence
A Practical Randomized CP Tensor Decomposition
A Practical Randomized CP Tensor Decomposition Casey Battaglino, Grey Ballard 2, and Tamara G. Kolda 3 SIAM CSE 27 Atlanta, GA Georgia Tech Computational Sci. and Engr. Wake Forest University Sandia National
Intro Vectors 2D implicit curves 2D parametric curves. Graphics 2012/2013, 4th quarter. Lecture 2: vectors, curves, and surfaces
Lecture 2, curves, and surfaces Organizational remarks Tutorials: TA sessions for tutorial 1 start today Tutorial 2 will go online after lecture 3 Practicals: Make sure to find a team partner very soon
Derived Algebraic Geometry III: Commutative Algebra
Derived Algebraic Geometry III: Commutative Algebra May 1, 2009 Contents 1 -Operads 4 1.1 Basic Definitions........................................... 5 1.2 Fibrations of -Operads.......................................
S. K. Mohanta, Srikanta Mohanta SOME FIXED POINT RESULTS FOR MAPPINGS IN G-METRIC SPACES
DEMONSTRATIO MATHEMATICA Vol. XLVII No 1 2014 S. K. Mohanta Srikanta Mohanta SOME FIXED POINT RESULTS FOR MAPPINGS IN G-METRIC SPACES Abstract. We prove a common fixed point theorem for a pair of self
a b = a a a and that has been used here. ( )
Review Eercise ( i j+ k) ( i+ j k) i j k = = i j+ k (( ) ( ) ) (( ) ( ) ) (( ) ( ) ) = i j+ k = ( ) i ( ( )) j+ ( ) k = j k Hence ( ) ( i j+ k) ( i+ j k) = ( ) + ( ) = 8 = Formulae for finding the vector
2. A die is rolled 3 times, the probability of getting a number larger than the previous number each time is
. If P(A) = x, P = 2x, P(A B) = 2, P ( A B) = 2 3, then the value of x is (A) 5 8 5 36 6 36 36 2. A die is rolled 3 times, the probability of getting a number larger than the previous number each time
The Riemann Roch theorem for metric graphs
The Riemann Roch theorem for metric graphs R. van Dobben de Bruyn 1 Preface These are the notes of a talk I gave at the graduate student algebraic geometry seminar at Columbia University. I present a short
Solutions of exercise sheet 3
Topology D-MATH, FS 2013 Damen Calaque Solutons o exercse sheet 3 1. (a) Let U Ă Y be open. Snce s contnuous, 1 puq s open n X. Then p A q 1 puq 1 puq X A s open n the subspace topology on A. (b) I s contnuous,
Review of Vector Analysis in Cartesian Coordinates
Review of Vector Analysis in Cartesian Coordinates 1 Scalar: A quantity that has magnitude, but no direction. Examples are mass, temperature, pressure, time, distance, and real numbers. Scalars are usually
Extra Problems for Math 2050 Linear Algebra I
Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as