Conservation of Information Amr Sabry (in collaboration with Roshan P. James) School of Informatics and Computing Indiana University May 8, 2012 Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 1 / 53
Physics and Computation Current models of computation were inspired by the laws of nature as understood in the past centuries. Quantum mechanics is currently the most accurate physical theory known to us. Let us investigate to what extent conventional models of computation are compatible with the (modern) laws of physics. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 2 / 53
Physical principles for computation and logic There are many principles we can embrace. We will start with one, and follow its consequences, which will turn out to be far reaching. Conservation of information Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 3 / 53
The Origin Ed Fredkin pursued the idea that information must be finite in density. One day, he announced that things must be even more simple than that. He said that he was going to assume that information itself is conserved. You re out of you mind, Ed. I pronounced. That s completely ridiculous. Nothing could happen in such a world. There couldn t even be logical gates. No decisions could ever be made. But when Fredkin gets one of his ideas, he s quite immune to objections like that; indeed, they fuel him with energy. Soon he went on to assume that information processing must also be reversible and invented what s now called the Fredkin gate. Marvin Minsky, Feynman and Cellular Vacuum, Feynman and Computation: Exploring the Limits of Computers, 1999 Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 4 / 53
Types and Type Isomorphims Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 5 / 53
Finite Types base types, b ::= 0 1 b + b b b Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 6 / 53
Finite Types base types, b ::= 0 1 b + b b b The axioms of a commutative semiring constitute a sound and complete set of type isomorphisms. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 6 / 53
Sound and Complete Isomorphisms 0 + b b identity for + b 1 + b 2 b 2 + b 1 commutativity for + b 1 + (b 2 + b 3 ) (b 1 + b 2 ) + b 3 associativity for + Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 7 / 53
Sound and Complete Isomorphisms 0 + b b identity for + b 1 + b 2 b 2 + b 1 commutativity for + b 1 + (b 2 + b 3 ) (b 1 + b 2 ) + b 3 associativity for + 1 b b identity for b 1 b 2 b 2 b 1 commutativity for b 1 (b 2 b 3 ) (b 1 b 2 ) b 3 associativity for Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 7 / 53
Sound and Complete Isomorphisms 0 + b b identity for + b 1 + b 2 b 2 + b 1 commutativity for + b 1 + (b 2 + b 3 ) (b 1 + b 2 ) + b 3 associativity for + 1 b b identity for b 1 b 2 b 2 b 1 commutativity for b 1 (b 2 b 3 ) (b 1 b 2 ) b 3 associativity for 0 b 0 distribute over 0 (b 1 + b 2 ) b 3 (b 1 b 3 ) + (b 2 b 3 ) distribute over + Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 7 / 53
Sound and Complete Isomorphisms 0 + b b identity for + b 1 + b 2 b 2 + b 1 commutativity for + b 1 + (b 2 + b 3 ) (b 1 + b 2 ) + b 3 associativity for + 1 b b identity for b 1 b 2 b 2 b 1 commutativity for b 1 (b 2 b 3 ) (b 1 b 2 ) b 3 associativity for 0 b 0 distribute over 0 (b 1 + b 2 ) b 3 (b 1 b 3 ) + (b 2 b 3 ) distribute over + b 1 b 1 b 1 b 2 b 2 b 1 b 1 b 2 b 2 b 3 b 1 b 3 b 1 b 3 b 2 b 4 (b 1 + b 2 ) (b 3 + b 4 ) b 1 b 3 b 2 b 4 (b 1 b 2 ) (b 3 b 4 ) Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 7 / 53
A Programming Language (Π) emerges... Add values for each type Add witnesses to the isomorphisms Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 8 / 53
A Programming Language (Π) emerges... Add values for each type Add witnesses to the isomorphisms base types, b ::= 0 1 b + b b b values, v ::= () left v right v (v, v) there are no values of type 0 () is the unique value of type 1 if v is of type b 1 then left v is of type b 1 + b 2 if v is of type b 2 then right v is of type b 1 + b 2 if v 1 is of type b 1 and v 2 is of type b 2 then (v 1, v 2 ) is of type b 1 b 2 Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 8 / 53
A Programming Language ( Π) emerges... zeroe : 0 + b b : zeroi swap + : b 1 + b 2 b 2 + b 1 : swap + assocl + : b 1 + (b 2 + b 3 ) (b 1 + b 2 ) + b 3 : assocr + unite : 1 b b : uniti swap : b 1 b 2 b 2 b 1 : swap assocl : b 1 (b 2 b 3 ) (b 1 b 2 ) b 3 : assocr distrib 0 : 0 b 0 : factor 0 distrib : (b 1 + b 2 ) b 3 (b 1 b 3 ) + (b 2 b 3 ) : factor id : b b c : b 1 b 2 sym c : b 2 b 1 c 1 : b 1 b 2 c 2 : b 2 b 3 (c 1 c 2 ) : b 1 b 3 c 1 : b 1 b 3 c 2 : b 2 b 4 (c 1 + c 2 ) : (b 1 + b 2 ) (b 3 + b 4 ) c 1 : b 1 b 3 c 2 : b 2 b 4 (c 1 c 2 ) : (b 1 b 2 ) (b 3 b 4 ) Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 9 / 53
Example The type isomorphism: (1 + 1) ((1 + 1) b) = (1 + 1) ((1 b) + (1 b)) = (1 + 1) (b + b) = (1 (b + b)) + (1 (b + b)) = (b + b) + (b + b) corresponds to the program: (id (distrib (unite unite))) (distrib (unite unite)) Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 10 / 53
Graphical notation Graphical language for symmetric monoidal categories. Values are waves or particles. Combinators are interconnections of wires. Computation is modeled as the flow of waves or particles on the wires. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 11 / 53
Graphical notation: Wires The simplest sort of diagram is the id : b b combinator which is simply represented as a wire labeled by its type b. When tracing a computation, one might imagine a value v of type b on the wire. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 12 / 53
Graphical notation: Products The product type b 1 b 2 may be represented using either one wire labeled b 1 b 2 or two parallel wires labeled b 1 and b 2. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 13 / 53
Graphical notation: Sums Sum types may similarly be represented by one wire or using parallel wires with a + operator between them. mr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 14 / 53
Graphical notation: Associativity Associativity is implicit in the graphical language. Three parallel wires represent b 1 (b 2 b 3 ) or (b 1 b 2 ) b 3, based on the context. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 15 / 53
Graphical notation: Commutativity Commutativity is represented by crisscrossing wires. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 16 / 53
Graphical notation: Distributivity and Factoring Distributivity and factoring are represented using the dual boxes shown below: Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 17 / 53
Programming in Π: Booleans, Negation We encode booleans by bool = 1 + 1 true = left () false = right () Negation: not : bool bool not = swap + Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 18 / 53
Programming in Π: Conditionals Reversible conditional: if c (flag, b) = if flag then (flag, c(b)) else (flag, b) Given c : b b, then: if c : bool b bool b if c = distrib ((id c) + id) factor Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 19 / 53
Programming in Π: Fredkin and Toffoli Gates Universal gates for reversible combinational circuits Fredkin gate: if swap Toffoli gate: if ifswap + Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 20 / 53
Information Effects Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 21 / 53
Closed vs. Open Systems So far we have modeled closed systems in which information is preserved; An open system may have interactions with its environment; We will show that these interactions can be modeled with two new operations that interact with the environment via side effects: create : 1 b erase : b 1 The effects are isolated using an arrow metalanguage which is a standard way of modeling computational effects. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 22 / 53
One slide on arrows Abstractly, computations are maps that can do side effects. Pure computations can be lifted to the world of arrows: arr : (b c) (b c) Computations can be sequenced: : (b c) (c d) (b d) Computations can be selectively applied to inputs (the rest of the input can be thought of as the environment that is generally not affected): first : (a b) ((b, d) (c, d)) Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 23 / 53
λ and information effects Conventional programming languages (λ ) have accidently embodied information effects! Translate from a first-order λ, with sum and product types, to the arrow language with create and erase: Γ e : b = a : Γ b Translate from the arrow language with create and erase to Π: a : Γ b = c : h Γ g b where h and g, exposed in the types, represent the implicit information environment. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 24 / 53
Example: XOR xor(b 1, b 2 ) = if b 1 then not b 2 else b 2 The most optimal implementation of xor must erase at least 1 bit i.e. there must be at least one erase bool. xor : bool bool bool xor = distrib (not id) factor (erase bool id) arr unite Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 25 / 53
Example: NAND nand(b 1, b 2 ) = if b 1 then not b 2 else true The most optimal implementation of nand must erase at least 1.2 bits i.e. at least two bools must be erased. nand : bool bool bool nand = distrib (not (erase bool create true )) factor (erase bool id) arr unite Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 26 / 53
Applications A better basis of computation than λ-calculus for applications where information manipulation is computationally significant: Quantitative information-flow security Differential privacy Energy-aware computing VLSI design Biochemical models of computation etc. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 27 / 53
So far... A programming language Π universal over finite types; Based on type isomorphisms; conservation of information; Can model creation, cloning, and erasure of information as side effects; Related to linear logic (no weakening and no contraction); quantum computing (no-cloning and no-deleting); How about recursion, higher-order functions, etc.? Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 28 / 53
Categorial Interlude Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 29 / 53
Categorical Perspective base types, b ::= 0 1 b + b b b The subset of Π with (, 1) is a symmetric monoidal category; The subset of Π with (+, 0) is another symmetric monoidal category; The full language Π is a category with two symmetric monoidal structures which satisfy distributivity of over + (but not vice-versa). Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 30 / 53
Compact Closed Categories Compact closed categories have a notion of higher-order functions; necessary for self-interpreters and universal computation. Compact closed categories have a canonical notion of trace; the categorical abstraction of feedback, looping, and recursion; also necessary for universal computation. Technically all we need is to extend the semidistributive symmetric monoidal structure of Π to a compact closed structure. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 31 / 53
The Int Construction (Joyal, Street, and Verity, 1996) Start from a traced monoidal category; Introduce a new class of arrows flowing backwards ; Define appropriate notions of composition, etc. and compact closure is guaranteed! Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 32 / 53
Key Insight Arrows flowing forward carry waves/particles from producers to consumers; Arrows flowing backwards carry demands from consumers to producers; A function can be modeled using a backwards arrow that demands the argument and a forward arrow that produces the result! Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 33 / 53
Negation and Symmetry Idea of values flowing backwards is appealing: From a logical perspective, it corresponds to some form of (involutive) negation; Frm a physical perspective, it introduces symmetry which is probably the most widely accepted physical principle. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 34 / 53
Good and bad news It is possible to independently extend each symmetric monoidal structure to a compact closed one; The two compact closed structures appear incompatible with each other; It doesn t appear possible to get one notion of negation like in propositional, classical, intuitionistic, and even linear logic; It doesn t appear possible to get one notion of functions like in most models of computation; Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 35 / 53
Solution Keep trying to find the unifying structure, or: Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 36 / 53
Solution Keep trying to find the unifying structure, or: Give up our preconceived notions about negation and functions, and investigate: two notions of negation; two notions of functions; two notions of duality; and two notions of symmetry! Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 36 / 53
Negative and Fractional Types Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 37 / 53
Extending the language Π ηɛ Types Isomorphisms Values base types, b ::= 0 1 b + b b b b 1/b η + : 0 ( b) + b : ɛ + η : 1 (1/b) b : ɛ v : b v : b v : b 1/v : 1/b mr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 38 / 53
Additive and Multiplicative Dualities Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 39 / 53
Additive and Multiplicative Dualities Backward information flow Creation and annihilation of entangled particles Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 40 / 53
Example Consider the isomorphism: a = a + 0 = a + ( a + a) = (a + ( a)) + a = 0 + a = a It corresponds to the program: mr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 41 / 53
Example It has a very interesting computational content: Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 42 / 53
Example It has a very interesting computational content: Semantically equivalent to the identity function, but models teleportation. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 42 / 53
Multiplicative verion Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 43 / 53
Multiplicative verion Semantically equivalent to the identity function, but models a credit card transaction. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 43 / 53
Semantics (Additives) Given a combinator c and a value v, we can propagate the value forward; (by convention in the diagrams this is the left-to-right flow of values). Since combinators represent isomorphisms, we can also propagate a value backwards; (by convention in the diagrams this is the right-to-left flow of values). A value v flows in the opposite direction of v; (and hence ( v) is the same as v). Formally we have two interpreters: a forward interpreter and a reverse interpreter. If one of the interpreters encounters a combinator whose output is the 0 type, it calls the other one. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 44 / 53
Semantics (Multiplicatives) At any point, during the execution, we can create a value/constraint pair α and 1/α; If either α or 1/α is used in a way that fixes α, the dual is immediately adjusted. Formally, we augment the interpreters with unification Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 45 / 53
(Additive) Trace Given f : b 1 + b 2 b 1 + b 3, define trace + f : b 2 b 3 as: trace + f = zeroi (id + η + ) (f + id) (id + ɛ + ) zeroe Similarly for trace. Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 46 / 53
(Additive) Functions: Definition Applying a function to an argument consists of making the argument flow backwards to satisfy the demand of the function: Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 47 / 53
(Additive) Functions: Composition which is essentially equivalent to sequencing both the computation blocks as shown below: Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 48 / 53
More constructions The same diagrams but with a different interpretation give rise to multiplicative functions; All the identities that hold in a mathematical field become isomorphisms with fascinating computational interpretations; Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 49 / 53
Conclusions Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 50 / 53
Negative and Fractional Types Make sense in a world in which information is conserved; Can borrow because the infrastructure guarantees that all debts must be accounted for; Negative and fractional types provide fascinating mechanisms in which computations can be sliced and diced, decomposed and recomposed, run forwards and backwards, in arbitrary ways; Enable greater flexibility in programming by breaking global invariants into local ones that can be autonomously satisfied by a subcomputation; Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 51 / 53
Algebraic Computation We started with Π that expresses computation in the commutative semiring of whole numbers Extended to Π ηɛ that expresses computation in the field of rationals Possible extension is to allow user-defined (recursive) datatype defintions: these are polynomials, i.e., we can compute in the field of algebraic numbers; Example: The type of binary trees µx.(1 + x x) can be rewritten as x = 1 + x x or x 2 x + 1 = 0, i.e., the type x is (1/2) + i( 3/2)! Confirms previous observations that imaginary and square root types make sense! Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 52 / 53
Symmetries, Dualities, of Computation Duality between values (forward flow) and continuations (backwards flow) was previously observed (1989); The principle of conservation of information unveils deeper and more elegant symmetries; Towards a computational understanding quantum mechanics? Amr Sabry (in collaboration with Roshan P. James) (IU SOIC) Conservation of Information May 8, 2012 53 / 53