Linear logic and deep learning. Huiyi Hu, Daniel Murfet

Size: px

Start display at page:

Download "Linear logic and deep learning. Huiyi Hu, Daniel Murfet"

Blanche Janis Hicks
6 years ago
Views:

1 Linear logic and deep learning Huiyi Hu, Daniel Murfet

2 Move 37, Game 2, Sedol-lphaGo It s not a human move. I ve never seen a human play this move So beautiful. - Fan Hui, three-time European Go champion.

3 Outline Introduction to deep learning What is differentiable programming? Linear logic and the polynomial semantics Our model

4 2 Intro to deep learning deep neural network (DNN) is a particular kind of function F w : R m! R n which depends in a differentiable way on a vector of weights. Example: Image classification, as in: cat or dog? 0.7 F w = R 2 R p cat =0.7 p dog =0.3

bars) were given four handicap stones (that is, free moves at the start of every game) against all opponents.

correspondence to human ranks is also shown, Intro to deep learning better in lphago than a value function vθ(s ) v pσ(s ) derived from the deep neural network (DNN) is a particular kind of function

R w efficiently combine MCTS with deep neural networks, lphago uses an asynchronous multi-threaded search that executes simulations on which depends in a differentiable way onnetworks a vector of

5 bars) were given four handicap stones (that is, free moves at the start of every game) against all opponents. Programs were evaluated on an Elo scale37: a 230 point gap corresponds to a 79% probability of winning, which roughly corresponds to one amateur dan rank advantage on KGS38; an approximate correspondence to human ranks is also shown, Intro to deep learning better in lphago than a value function vθ(s ) v pσ(s ) derived from the deep neural network (DNN) is a particular kind of function SL policy network. Evaluating policy and value networks requires several orders of m n traditional search heuristics. To magnitude more computation than F : R! R w efficiently combine MCTS with deep neural networks, lphago uses an asynchronous multi-threaded search that executes simulations on which depends in a differentiable way onnetworks a vector of weights. CPUs, and computes policy and value in parallel on GPUs. The final version of lphago used 40 search threads, 48 CPUs, and 8 GPUs. We also implemented a distributed version of lphago that Example: Playing the game of Go a Value network explo 176 G and d Eval To ev of l comm sourc b Tree evaluation from 2R Fw (current board) = comb does n with s distri R19 19 d Policy network e Percentage g of simul

6 Intro to deep learning deep neural network (DNN) is a particular kind of function F w : R m! R n which depends in a differentiable way on a vector of weights. randomly initialised neural network is trained on sample data {(x i,y i )} i2i,x i 2 R m,y i 2 R n by varying the weight vector in order to minimise the error E w = X i (F w (x i ) y i ) 2

7 training process (gradient descent) E w initial network w 1 w 2 optimal network

8 Intro to deep learning DNN is made up of many layers of neurons connecting the input (on the left) to the output (on the right) as in the following diagram: x 1 y 1 w 1j w 11 x j y i w ij y i = X w ij x j (x) = 1 2 (x + x ) j

9 Intro to deep learning DNN is made up of many layers of neurons connecting the input (on the left) to the output (on the right) as in the following diagram: (k) (k + 1) w (k) 11 x 1 y 1 w (k) 1j x j w (k) ij y i y i = X j w (k) ij x j (x) = 1 2 (x + x )

10 Intro to deep learning DNN is made up of many layers of neurons connecting the input (on the left) to the output (on the right) as in the following diagram: (k) (k + 1) w (k) 11 x 1 y 1 w (k) 1j x j w (k) ij y i y i = X j w (k) ij x j ~y = W (k) ~x (x) = 1 2 (x + x ) ~x =(x 1,x 2,...,x j,...) T W (k) =(w (k) ij ) i,j

11 Intro to deep learning DNN is made up of many layers of neurons connecting the input (on the left) to the output (on the right) as in the following diagram: x 1 (0) (L + 1) x j x m F w : R m! R n F w (~x) = W (L) W (1) W (0) ~x

12 pplied deep learning Natural Language Processing (NLP) includes machine translation, processing of natural language queries (e.g. Siri) and question answering wrt. unstructured text (e.g. where is the ring at the end of LoTR? ). Two common neural network architectures for NLP are Recurrent Neural Networks (RNN) and Long-Short Term Memory (LSTM). oth are variations on the DNN architecture. Many hard unsolved NLP problems require sophisticated reasoning about context and an understanding of algorithms. To solve these problems, recent work equips RNNs/LSTMs with Von Neumann elements e.g. stack memory. RNN/LSTM controllers + differentiable memory is one model of differentiable programming, currently being explored at e.g. Google, Facebook,

Salon, Facebook Thinks It Has Found the Secret to

13 These memory enhanced neural networks are already being used in production, e.g. Facebook messenger (~900m users). Salon, Facebook Thinks It Has Found the Secret to Making ots Less Dumb, June

14 What is differentiable programming? The idea is that a differentiable program P w : I! O should depend smoothly on weights, and, by varying the weights, one would learn a program with desired properties. Question: what if instead of coupling neural net controllers to imperative algorithmic elements (e.g. memory) we couple a neural net to functional algorithmic elements? Problem: how to make an abstract functional language like Lambda calculus, System F, etc. differentiable? Our idea: use categorical semantics in a category of differentiable mathematical objects compatible with neural networks (e.g. matrices of polynomials).

15 Linear logic and semantics Discovered by Girard in the 1980s, linear logic is a substructural logic with contraction and weakening available only for formulas marked with a new connective! (called the exponential). The usual connectives of logic (e.g. conjunction, implication) are decomposed into! together with a linearised version of that connective (called resp. tensor, linear implication ). ( Under the Curry-Howard correspondence, linear logic corresponds to a programming language with resource management and symmetric monoidal categories equipped with a special kind of comonad.

16 Linear logic and semantics variables:,,,... formulas:!f, F F 0,F ( F 0,F&F 0, 8 F, constants int = 8!( ( ) ( ( ( ) bint = 8!( ( ) (!( ( ) ( ( ( )

17 ! all deduction rules, the sets and may be empty and, in particular, the promotion C (xiom): for a list of ` may be used with an empty premise. In the promotion rule,! stands ulas each of which is preceeded by an exponential modality, for examplelogic! 1,...,!n. Deduction rules for linear 0 0 ( ` ` `,, ` diagrams on the right are string diagrams and should be ignored until Section 5. In -R (Right ): (4.5),,, `C cut C (Cut): 0, `! ` -L (4.6) ):trees associated prom,, ` (4.9) (Promotion): cular they are(left not the,! `!, ` to C proofs.! (xiom): ` ` 0,, (Left (): 0,, (, (Left ): (Cut): `C (L `C (Promotion):,,, ` C,, 0 ` C,, ` (Dereliction):,,, ` C,,!, `` (Exchange): (Right ():,,, `,, ` cut C (Exchange): (4.2) 0,, ` (4.8)! `! 0 der (R ` C `(Right ( ): -L! `! prom C (Right ): ( ( 0 ` `, ` -R,,,,,, `C `C 0 ( (4.9) C (4.6)! (4.10) 0 `,, ` C! (4.3) ` (Left`(): -R 0,, (, `C ( L (4.7), ` (! C ( 8 Definition 7.5 (Second-order linear logic). The Definition 7.5 (Second-order linear logic). The formulas of second-order linear logic, formula `,, ` recursively! `,,, ` C are defined as follows: any pr prom weak of (Weakening): (4.9) are defined recursively as follows: any formula of (Left propositional linear logic is -L a formula, (Promotion): der (4.10 (Dereliction): ):,!, 0 `! `!,!, `,, ` C and if is a formula then so is 8x. for any proposi and if is a formula then so is 8x. for any propositional variable x.there are two new C deduction rules: deduction rules:, ` (4.7) (R (Right ():!! ```(8R, [/x] ` C `,!,!,, 8R 8L (4.11) (Contraction): ` 8x. ctr, 8x ` 8x.,!, `,,!, ` C. ` C, 8 -L ( (Left ):, empty,, `and C in the right introduction rule we ( where is a sequence of formulas,8possibly a is a sequence of formulas, possibly empty, require that x is not free in any formula of where. Here [/x] means a formula with all, `! 1-L (Left 1):! ` prom that x is not free in any formula of. Here free occurrences of x replaced by a formula require (as usual, there is some chicanery necessary (Promotion):, 1, `,!,!, `! `! ctr (4.11 (Contraction):

18 inary integers bint = 8!( ( ) (!( ( ) ( ( ( ) S 2 {0, 1} 7! proof t S of ` bint ` ` ( L `, ( ` ( L `, (, ( ` ( L, (, (, ( ` ( R (, (, ( ` ( der t 001!( ( ),!( ( ),!( ( ) ` ( ctr!( ( ),!( ( ) ` (!( ( ) `!( ( ) ( ( ( ) ( R `!( ( ) (!( ( ) ( ( ( ) ( R

19 Polynomial semantics (sketch) J K = R d J ( K = L(R d, R d )=M d (R) J!( ( )K = R[{a ij } 1applei,jappled ]=R[a 11,...,a ij,...,a dd ] (one variable per entry in a d x d matrix) JbintK = J!( ( ) (!( ( ) ( ( ( ) K = M d R[{a ij,a 0 ij} 1applei,jappled ] (matrices of polynomials in the variables a, a )

20 Polynomial semantics (sketch) JbintK = M d R[{a ij,a 0 ij} 1applei,jappled ] (matrices of polynomials in the variables a, a ) S 2 {0, 1} t S : bint, Jt S K 2 JbintK Jt 0 K = 0 a 11 a a 21 a 2d a d1 a dd 1 C Jt 1K = 0 a 0 11 a 0 a 0 21 a 0 2d a 0 d1 a 0 dd 1 C Jt 001 K = Jt 0 K Jt 0 K Jt 1 K

21 Polynomial semantics (sketch) JbintK = M d R[{a ij,a 0 ij} 1applei,jappled ] (matrices of polynomials in the variables a, a ) t S : bint, Jt S K 2 JbintK vector space Jt 0 K Jt 1 K Jt 001 K JbintK Upshot: proofs/programs to vectors

22 Polynomial semantics (sketch) T 2 JbintK = M d R[{a ij,a 0 ij} 1applei,jappled ] (matrices of polynomials in the variables a, a ) eval T (, ) =T aij = ij,a 0 ij = ij, 2 M d (R) Jt 0 K Jt 1 K T Jt 001 K JbintK Upshot: proofs/programs to vectors

23 Polynomial semantics (sketch) T 2 JbintK = M d R[{a ij,a 0 ij} 1applei,jappled ] (matrices of polynomials in the variables a, a ) eval Jt0 K(, ) = eval Jt001 K(, ) = Jt 0 K Jt 1 K T Jt 001 K JbintK Upshot: proofs/programs to vectors

... > ' Our model: RNN controller + differentiable linear logic module H ' ) Q 7 i H ) S ) - : eval,h( TMH ), ; Q QT - ' i hit tit Y - -1 9.

24 ... > ' Our model: RNN controller + differentiable linear logic module H ' ) Q 7 i H ) S ) - : eval,h( TMH ), ; Q QT - ' i hit tit Y Tlt ) U : > " ; : ;Ct ) T (t) 2 JbintK H, S, U, Q,Q,Q T are matrices of weights (t) = h (t) = S eval T (t)( (t), (t) )+Hh (t 1) + Ux (t) Q h (t 1) (t) = Q h (t 1) T (t) = (t Q T h 1)

Stratifications and complexity in linear logic. Daniel Murfet

Stratifications and complexity in linear logic Daniel Murfet Curry-Howard correspondence logic programming categories formula type objects sequent input/output spec proof program morphisms cut-elimination