Semantic Frameworks for Implicit Computational Complexity (Ph.D. Thesis)

Transcription

1 Semantic Frameworks for Implicit Computational Complexity (Ph.D. Thesis) Ugo Dal Lago Technical Report UBLCS March 2006 Department of Computer Science University of Bologna Mura Anteo Zamboni Bologna (Italy)

2 The University of Bologna Department of Computer Science Research Technical Reports are available in PDF and gzipped PostScript formats via anonymous FTP from the area ftp.cs.unibo.it:/pub/tr/ublcs or via WWW at URL Plain-text abstracts organized by year are available in the directory ABSTRACTS. Recent Titles from the UBLCS Technical Report Series Interactivity Maintenance for Event Synchronization in Massive Multiplayer Online Games (Ph.D. Thesis), Ferretti, S., March Reasoning with preferences over temporal, uncertain, and conditional statements (Ph.D. Thesis), Venable, K. B., March Whole Platform (Ph.D. Thesis), Solmi, R., March Loss Functions and Structured Domains for Support Vector Machines (Ph.D. Thesis), Portera, F., March A Reasoning Infrastructure to Support Cooperation of Intelligent Agents on the Semantic Grid, Dragoni, N., Gaspari, M., Guidi, D., April Fault Tolerant Knowledge Level Communication in Open Asynchronous Multi-Agent Systems, Dragoni, N., Gaspari, M., April The AEDSS Application Ontology: Enhanced Automatic Assessment of EDSS in Multiple Sclerosis, Gaspari, M., Saletti, N., Scandellari, C., Stecchi, S., April How to cheat BitTorrent and why nobody does, Hales, D., Patarin, S., May Choose Your Tribe! - Evolution at the Next Level in a Peer-to-Peer network, Hales, D., May Knowledge-Based Jobs and the Boundaries of Firms: Agent-based simulation of Firms Learning and Workforce Skill Set Dynamics, Mollona, E., Hales, D., June Tag-Based Cooperation in Peer-to-Peer Networks with Newscast, Marcozzi, A., Hales, D., Jesi, G., Arteconi, S., Babaoglu, O., June Atomic Commit and Negotiation in Service Oriented Computing, Bocchi, L., Ciancarini, P., Lucchi, R., June Efficient and Robust Fully Distributed Power Method with an Application to Link Analysis, Canright, G., Engo-Monsen, K., Jelasity, M., September On Computing the Topological Entropy of One-sided Cellular Automata, Di Lena, P., September A model for imperfect XML data based on Dempster-Shafer s theory of evidence, Magnani, M., Montesi, D., September Friends for Free: Self-Organizing Artificial Social Networks for Trust and Cooperation, Hales, D., Arteconi, S., November Greedy Cheating Liars and the Fools Who Believe Them, Arteconi, S., Hales, D., December Lambda-Types on the Lambda-Calculus with Abbreviations: a Certified Specification, Guidi, F., January On the Quality-Based Evaluation and Selection of Grid Services (Ph.D. Thesis), Andreozzi, S., March Transactional Aspects in Coordination and Composition of Web Services (Ph.D. Thesis), Bocchi, L., March 2006.

3 Semantic Frameworks for Implicit Computational Complexity (Ph.D. Thesis) Ugo Dal Lago Technical Report UBLCS March 2006 Abstract This thesis is about computational complexity, programming languages and mathematical logic. The main contribution of this thesis is the study of two unifying frameworks for the analysis of complexity properties of higher-order programs and proofs. The first one is Gonthier, Abadi and Lévy s context semantics, which is shown to be applicable to the quantitative analysis of proofs and programs. A new notion of context semantics for multiplicative and exponential linear logic is introduced and related to the complexity of normalizing proofs. Moreover, a context semantics for higher-order primitive recursion is studied, obtaining some novel characterization results for fragments of the calculus. The second one is a framework based on realizability models. It leads to new proofs of soundness for three different subsystems of linear logic. As a preliminary step, a new invariant cost model for the pure, call-by-value lambda calculus is defined. Semantical models described in this thesis are modifications (or generalizations) of already known models. Nonetheless, here they are shown to be applicable to the quantitative analysis of a wide range of systems. This leads to some interesting new results. 1

4

5 Chapter 1 Introduction This thesis deals with computational complexity, programming languages and mathematical logic. The relationships between these three disciplines are indeed very deep. Recursion theory is a branch of mathematical logic studying the intrinsic computational difficulty of functions. Through the so-called subrecursive classes [Ros84], it identifies hierarchies of recursive functions taking into account various attributes of the algorithms computing them: their length, their structure and the amount of resources (time and space) they need. Computational complexity, in particular, can be defined as the study of small subrecursive classes based on time and space. Indeed, a pioneering recursion theoretical work by Cobham [Cob65] is one of the first attempts to relate polynomial time computable functions and feasibility. Systematically studying tools for classifying programs and functions with respect to their computational complexity is relevant for programming languages, too. While programming languages are usually very expressive, there are situations where knowing the amount of resources a program consumes is crucial, typical examples being real-time systems and safety-critical systems. Proof theory, another branch of mathematical logic, studies properties of proofs and has been connected to programming languages since the discovery of the Curry-Howard Correspondence [How80]. The correspondence between programs and proofs is central to linear logic [Gir87], a refinement of classical logic which allows to control the complexity of cut-elimination by restricting the use of structural rules in derivations. The research area at the intersection of mathematical logic, computational complexity and programming language theory is known as implicit computational complexity and has produced many interesting systems in recent years, from function algebras [BC92, Lei93, LM94] to logical systems [Gir98, Laf04] to type systems [Hof99b, BT04]. In particular, many proposals can be traced back to either recursion theory or linear logic. All these systems are shown to extensionally correspond to computational complexity classes: the set of functions which are representable in the system equals the complexity class. However, proofs of these correspondences are usually ad-hoc and do not give any information on the relative intensional expressive power of systems: are there systems capturing every algorithm whose running time is inside a given class? How far can we go in this direction? Being able to deal with the above issues is crucial to the field, as far as the impact to real programming languages is concerned. Recently, there have been attempts to define systems extensionally corresponding to complexity classes and catching natural algorithms. However, it is not yet clear how relative intensional expressive powers relate one to the other. Moreover, there is no general criteria to evaluate the impact of extending a certain system with new features. To clarify these concepts, let us now consider an illustrative example: let sort : N N be a function which receives as input a finite sequence l of natural numbers and outputs an non-decreasing permutation of l. We can distinguish at least three different polynomial time algorithms computing sort (see [CLRS01] for a reference): First of all, we can iterate a conditional swapping operation a quadratic number of times, in the style of BubbleSort. 3

6 1 Contributions We can iterate an insertion algorithm a linear number of times. The insertion algorithm is itself defined iteratively and takes a linear amount of time. This algorithm is known as InsertionSort. We can partition the input sequence l into two subsequences f and s such that any element of f is smaller or equal to any element of s. We then apply recursively the same algorithm to f and s and concatenate the two results. This algorithm is known as QuickSort. The first two algorithms can be written in a functional programming language, provided it allows iteration. While most of the systems capturing polynomial time admit the first algorithm as a legal definition, many of them do not allow nested iterations. As a consequence, Insertion- Sort is usually rejected, although some systems captures it [Hof99a]. The situation is even worse for QuickSort, because recursion is not structural and the algorithm being polytime critically depends on size considerations about the partition step. Indeed, as far as the author knows, the only system which allows Quicksort as a legal definition is an extension of Hofmann s non-sizeincreasing computation [Hof03]. This example should clarify the meaning of intensional and extensional in this context: in extensionally complete systems at least one algorithm computing sort can be written, while in intensionally complete systems one is able to write every polytime algorithm, including all those computing sort. Now, please notice that we cannot reach intensional completeness, at least if we are interested in recursively enumerable systems: the class of polytime programs is not recursively enumerable as soon as the underlying programming language becomes Turing complete. This, however, makes the question about how far we can go toward intensional completeness even more interesting. A first step forward dealing with these challenges certainly consists in investigating on powerful semantical frameworks for the analysis of quantitative properties of proofs and programs. This is precisely what this thesis is aiming at. 1 Contributions The main contribution of this work is the study of two unifying frameworks for the analysis of complexity properties of higher-order programs and proofs. Part I presents some background information on implicit computational complexity and some preliminary definitions and results. In Part II, context semantics is shown to be applicable to the quantitative analysis of proofs and programs. A new notion of context semantics for multiplicative and exponential linear logic is introduced in Chapter 4, followed by some results on its relation to complexity. Chapter 5 is devoted to context semantics for higher-order primitive recursion. Some novel characterization results for fragments of the calculus follow. A framework based on realizability models is the subject of Part III. As a preliminary step, a new cost model for the pure, call-by-value lambda calculus is introduced in Chapter 6. The framework itself is described in Chapter 7, together with new proofs of soundness for three different subsystems of linear logic. Semantical models described in this thesis are modifications (or generalizations) of already known models. Nonetheless, here they are shown to be applicable to the quantitative analysis of a wide range of systems. This leads to some interesting new results, namely the extensional characterization results in Chapter 5. Technical contributions of this thesis can be found in some preprints [DLH05a, DL05b, DL05a, DLM05] and two recently published extended abstract [DL05c, DLH05b]. Some of them are joint works with Simone Martini and Martin Hofmann. Other contributions to implicit computational complexity by the author [DLMR03, DL03, DLM04, DLB04, CDLRDR05] are not included in this thesis, although they have been carried out during his doctoral studies. UBLCS

7

8 Part I Motivations and Preliminaries UBLCS

9 Chapter 2 Implicit Computational Complexity and Semantics Implicit computational complexity studies theory and applications of machine-independent approaches to computational complexity, with particular emphasis on approaches based on mathematical logic. Indeed, different branches of logic have been proved to be useful in characterizing complexity classes. 1 Recursion Theory Recursion theory is a branch of mathematical logic which studies classes of functions and their intrinsic computational difficulty. An ubiquitous theme in recursion theory is that of a function algebra, which is the smallest class of functions containing certain initial functions and closed under certain operators. The most important operators in recursion theory are primitive recursion, function composition and function minimization. Since the fourties, many classes of total functions have been defined by somehow restricting primitive recursion scheme. In particular, we remind pioneering works by Kalmár and Grzegorczyk [Grz53]. An essential breakthrough towards complexity theory is due to Cobham [Cob65], who claimed polynomial time computable functions to be extensionally equivalent to a function algebra closed under bounded recursion on notation: a function f : N n+1 N is defined by bounded recursion on notation from g 0, g 1 : N n N, h 0, h 1 : N n+2 N and k : N n+1 N if f(0, y) f(1, y) f(2x, y) f(2x + 1, y) = g 0 (y); = g 1 (y); = h 0 (x, y, f(x, y)); = h 1 (x, y, f(x, y)); provided f(x, y) k(x, y). Bounded recursion on notation, together with bounded recursion and bounded minimization, has led to a variety of different characterizations of sequential, parallel and boolean circuit complexity classes (a good reference is [Clo99]). Checking a recursive definition for boundedness is not easy. On the other hand, even very weak function algebras can generate all the primitive recursion schemes if bounding conditions are dropped. This issue has been addressed by defining unbounded recursion schemes which control the growth of functions by partitioning function arguments into suitable classes. Function definitions are then constrained to somehow respect this partition. The first system pointing in this direction is due to Bellantoni and Cook [BC92]. In their setting, the arguments to a function f : N n N are partitioned into m n normal arguments and n m safe arguments. Without losing generality, normal arguments are assumed to be the ones on the left. In this way, f can be 7

10 2 Proof Theory denoted as f(x 1,..., x m ; x m+1,...,x n ), where the semicolon serves as a separator. The function f is defined by safe composition from g, h 1,..., h n, k 1,..., k m if f(x; y) = g(u 1 (x; ),...,u n (x; ); v 1 (x; y),...,v m (x; y)). The function f is defined by safe recursion on notation from g 0, g 1, h 0, h 1 if f(0, x; y) f(1, x; y) f(2x, x; y) f(2x + 1, x; y) = g 0 (x; y); = g 1 (x; y); = h 0 (x, x; y, f(x, x; y)); = h 1 (x, x; y, f(x, x; y)). Theorem 1 (Bellantoni and Cook [BC92]) The class FPTIME of polynomial time computable functions equals the class of functions definable by safe recursion and safe composition from constant 0, projections, successors, binary predecessor and conditional. Other complexity classes have been characterized using variations on the safe recursion scheme [Bel92]. Related to safe recursion is the notion of tiering, which has been exploited by Daniel Leivant to characterize FPTIME [Lei93, Lei95b] as well as polynomial space [LM94]. A survey on these results can be found in [Clo99]. 2 Proof Theory Proof theory studies how proofs are used in logic and, generally, in mathematics. Proofs, in particular, are studied as formal objects, without any reference to an intended semantics. Proof theory has strong relationships with computational complexity. According to the Curry-Howard correspondence, proofs can be seen as programs (in suitable programming languages), while proof normalization can be seen as program execution. Therefore, restricting a logical system implies restricting the expressive power of the underlying programming language. In this way, a logic can be shown to correspond to a complexity class, once the class of functions which are representable inside the logic is proved to be equal to the complexity class itself. However, what is a representation of a function inside a logic? If the logic is first-order arithmetic, the class of number-theoretic functions that can be proved recursive in the logic is a reasonable measure of the computational strength of the logical theory itself. In this sense, subsystems of arithmetic can be put in correspondence with computational complexity classes. The class of functions which are provably recursive in ordinary arithmetic includes functions which are so fast growing as to be very far from being realistically computable. Bounded arithmetic [Bus86] is an arithmetical system, introduced by Buss and inspired by Cobham s bounded recursion on notation [Cob65], whose provably recursive functions are exactly the polynomial time computable functions. Many other systems characterizing polynomial time in the same sense, but avoiding explicit bounding conditions, have been proposed [Nel86, Lei95a, Bel98, BH02]. Larger computational complexity classes (e.g. elementary time computable functions) have been found to correspond to stronger arithmetical theories [Lei95a, WO02]. If we consider logics without pre-interpreted symbols, the situation is apparently more difficult. In particular, it is not even clear how to represent inputs to functions. Consider, for example, intuitionistic propositional logic or, equivalently, simply-typed lambda-calculus. If we fix normal forms for A 0 = (α α) (α α) to be the only legal encoding of numerals, then the class of representable functions is very small (i.e. coincides with the class of extended polynomials [Sch73]). If we relax this constraint, for example by allowing alternative encodings for numerals as normal forms for A 1 = (A 0 A 0 ) (A 0 A 0 ), we increase the class of representable functions. Going to the limit, we find that normalization of a generic simply-typed lambda-term is not even elementary in the size of the term being normalized [Sta79, Mai92]. UBLCS

11 3 Model Theory Introducing second-order universal quantification allows to avoid all the problems we just mentioned. Indeed, if numerals are assumed to be encoded by normal forms for α.(α α) (α α), then we have: Theorem 2 (Representation Theorem) The set of representable functions for second-order lambdacalculus equals the class of functions that are provably total in second-order Peano arithmetic. Moreover, the above result is tight, that is to say strong normalization of second-order lambda-calculus (or second-order intuitionistic logic) is not provable by second-order Peano arithmetic. Unfortunately, provably total functions of second-order arithmetic are too strong for our purposes (i.e. no reasonable complexity class contains all of them). In the eighties, Girard introduced a new logic called linear logic [Gir87]. The main novelty of linear logic over intuitionistic logic is an explicit control over the use of structural rules: weakening and contraction can only be performed on exponential formulae, which are formulae in the form!a, where! is a new modal operator. It turns out that these rules, contraction in primis, are responsible for the unacceptable complexity of cut-elimination in intuitionistic logic. This observation has led many researchers to define fragments of linear logic and to prove remarkable properties on the complexity of cut-elimination for them. This effort has been made simpler by the availability of new syntactical objects representing proofs, namely proof-nets [Gir87, Gir95], which are very handy in studying and understanding the dynamics of proofs. The first attempt in this direction is bounded linear logic [GSS92]. In this system, occurrences of modal operator! and of atoms are labelled with certain polynomials, called resource polynomials. Logical rules use this information to force a polytime behavior. Indeed, the system can be proved to exactly capture polynomial time computable functions. Subsequently, Girard proposed another, simpler, system called light linear logic [Gir98]. Two modal operators,! and, are needed. However, they are not labelled with resource polynomials anymore. A very strong result can now be proved: Theorem 3 (Polytime soundness [Gir98]) The time needed to normalize a light linear logic proof-net of a lazy sequent is a polynomial on the size of the proof-net, provided the box-depth of proof-nets is fixed. A lazy sequent is a sequent not containing existential quantification or additive conjunction. The box-depth of a proof-net is a purely geometrical attribute of the proof-net. In particular, usual data-types such as natural numbers can be encoded by proof-nets with bounded box-depth in such a way that: Theorem 4 (Polytime completeness [Gir98]) Every polynomial time function on natural numbers is representable in light linear logic. Many different systems have been designed using the principles underlying light linear logic. We here recall elementary linear logic [Gir98, DJ03], which is a generalization of light linear logic capturing elementary time computable functions. Light affine logic [Asp98] simplifies light linear logic by making the weakening rule applicable to all formulae; in this way, additive logical operators are not needed anymore and the underlying syntax becomes considerably simpler. Light affine logic has been the subject of many syntactical and semantical investigations [AR02, KOS03, NM02, MO00a, MO00b, Ter02, DL03, DLB04]. Recently, Lafont proposed another very simple system, called soft linear logic, capturing polynomial time [Laf04]. In this system, there is just one modal operator!, while a strong notion of polytime soundness can be proved. 3 Model Theory Model theory is the study of mathematical objects underlying logical languages. Finite model theory focuses on models having finite cardinality. The restriction to finite models is not trivial, since there are formulae which are valid over finite structures but not over arbitrary structures (consider, as an example, the sentence if < is a linear order, then it has a maximum element ). UBLCS

12 4 From Logic to Programming Languages As a consequence, proof systems that are complete in the general case cannot prove these sentences. Noticeably, many corollaries of completeness (for example, the compactness theorem) are blatantly wrong in the world of finite models. Finite model theory has strong connections to computational complexity. In particular, many complexity classes can be proved to precisely correspond to the class of finite models of certain logical formulae. The first result in this direction is due to Fagin and goes back to 1973 [Fag73]. First, observe that a set F of finite models for a first-order language over a finite vocabulary can be considered as a language Enc(F), since a finite model can be easily encoded by strings over a finite alphabet. Then, define a Σ 1 1 sentence as a second-order formula consisting of a first-order formula where some of the predicate symbols have been existentially quantified out. The set of all finite models satisfying a Σ 1 1 formula is called a generalized spectrum. Then we have: Theorem 5 (Fagin [Fag73]) Let F be a set of finite models for a first-order language over a finite nonempty vocabulary. Suppose F is closed under isomorphism. Then Enc(F) is in the complexity class NPTIME if and only if F is a generalized spectrum. The above result allows to reduce very important open problems in computational complexity to a mathematical logic problem. The following result is again by Fagin: Theorem 6 (Fagin [Fag73]) NPTIME is closed under complement if and only if the class of generalized spectra is closed under complement. In the last thirty years, finite model theory has produced an impressive amount of results. Many of them are part of so-called descriptive complexity, strongly relating complexity classes to sets of models for certain logical theories (good references are [EF99], [Imm99], [Lib04] or [Fag93]) 4 From Logic to Programming Languages In the previous section, we briefly described many results about implicit computational complexity. Although the majority of them are actually ascribed to mathematical logic, their relevance to programming languages cannot be neglected. Consider, for example, Bellantoni and Cook s system: it can be regarded as a FPTIME-sound-and-complete first-order functional programming language. A similar situation occurs when the Curry-Howard correspondence on, say, light affine logic is considered. However, this is definitely not the end of the story. Modern functional programming languages (e.g. ML, Haskell, OCAML) offer many useful features for the definition of algorithms and data structures. This notwithstanding, most results on implicit computational complexity cannot be easily extended outside the simple, restrictive, framework in which they are formulated. Moreover, although most systems are extensionally complete for reasonably large complexity classes, intentional completeness does not hold: what is guaranteed is just the existence of one representation (in the system) of every function in the complexity class. Programming in these systems, in other words, can be a nightmare. Later on, we will briefly outline some results on resource-aware programming languages, focusing on type-based approaches. As already mentioned, function algebras can be considered as first-order programming languages. Much work has been done to study the applicability of concepts described in section 1 to a higher-order setting. Cook and Urquhart [CU93] embedded bounded recursion on notation into simply typed lambda calculus, showing that this does not change the class of definable first-order functions. A similar work, starting from safe recursion on notation, has been done by Hofmann, using modal types to reflect distinctions between safe and normal variables [Hof97]. If the recursion itself becomes higher-order, the expressive power of the language increases. For example, higher-order generalizations of Leivant s ramified recurrence captures elementary time computable functions [Lei99b]. Polynomial time can be retrieved by constraining higherorder variables to be used in a linear way. Variations of Gödel s system T capturing polynomial time by a combination of modal types and linear constraints have been given by Hof- UBLCS

13 4 From Logic to Programming Languages mann [Hof00] and Bellantoni, Niggl and Schwichtenberg [BNS00, BS01]. Hofmann then proposed a calculus for polynomial time functions which uses a stricter notion of linearity, but without any ramification condition [Hof99a]. Daniel Leivant has characterized major complexity classes using syntactical (but not type-theoretical) constraints on lambda-calculi with higher-type recursion [Lei99a]. Terui [Ter01, Ter02] proposed an (untyped) lambda calculus, called light affine lambda calculus, embedding ideas behind light affine logic into a programming language. Recently, Baillot and Mogbil made a similar work starting from Lafont s soft linear logic [BM04]. Light affine logic can be used as a type system for pure, untyped lambda-calculus, but will require some further work [BT04]. Elementary affine logic, the affine variants of elementary linear logic, has been proved to have strong connections to optimal reduction of functional programming languages [AG98]. Lambda terms which are typable in elementary affine logic do not require the full optimal reduction algorithms to be normalized. Indeed, the abstract algorithm (which is much more efficient than the full one) is sufficient. This observation has stimulated research on type-inference algorithms for elementary affine logic [CM01, CR03] and has led to important results on the complexity of optimal reduction [ACM00]. In the realm of (first-order) term-rewriting systems, techniques traditionally used to prove termination have been shown to be useful for inferring complexity properties of programs [BCMT99, BMM04]. Noticeably, many interesting algorithms are not rejected. Many other approaches to resource-awareness in programming languages have (apparently) nothing to do with implicit computational complexity. The following are just a few examples. Crary and Weirich [CW00] designed a framework for checking (but not inferring) resource bounds in the framework of typed assembly language. Benzinger [Ben01] introduced an approach to automated computational complexity analysis for higher-order functional programs in the context of program synthesis. Liu and Gomez [LG01] have used cost functions for automatically analyzing the time complexity of functional programs. 4.1 An Application: Resource-Awareness and Code Mobility In recent years, the use of mobile code has increased enormously. A new computation paradigm, called grid computing, is entirely based on code and data mobility. Code mobility often involves computing devices with limited resources, serving as host machines for mobile code. How can a host machine assure the amount of resource needed to run a mobile program? A resource-aware type system or program-logic would provide implicit and verifiable certificates. This approach does not rely on any previously established trust-network, in contrast to conventional security techniques based on extrinsic properties of programs (i.e. their origin). Computer science community has proved to be interested in these issues: many research projects have recently focused on programming language technologies for code mobility, with particular emphasis on security and efficiency issues 1. A number of different type systems and program logics exhibiting some kind of resource-awareness have recently appeared in the literature [HJ03, BHMS04]. Enforcing resource-awareness in programming languages is not an easy task. The additional control provided cannot come at the price of unacceptable restrictions to programs. Bounds on resource consumption should be reasonably tight. Last but not least, the problem of inferring resource bounds should remain decidable, if not feasible. These goals, however, can hardly be achieved by the same system. In other words, some trade-offs are necessary between programming flexibility, tightness of bounds and efficiency of program analysis. 1. Examples are the European project called Mobile Resource Guarantees (MRG, project IST ), the french project Contrôle de Ressources et d Interférence dans les Systèmes Synchrones (CRISS) and the project called Certified Code for Grid Computing (CONCERT) UBLCS

14 5 From Logic to Computational Complexity 5 From Logic to Computational Complexity Besides applications, there are strong theoretical motivations behind the interest in implicit computational complexity. Programming languages can be designed so that functions computable by acceptable programs extensionally correspond to certain computational complexity classes. If the underlying programming language is reasonably abstract, the system is then a machinefree characterization of a complexity class and can be used to infer properties of that same class. In this respect, take note that a number of different problems inside complexity theory still remain open, thirty years after their original formulation, suggesting that a deep mathematical theory is needed to make progress in this area [Laf03]. This area has many connections to mathematical logic, and specially to proof theory. The advent of Linear Logic [Gir87], in particular, has leveraged research on the interplay between proof theory and computational complexity: a number of different systems, all derived from Linear Logic and collectively called light logics, have been defined and proved to strongly correspond to different time complexity classes via the Curry-Howard correspondence. Recursion theory, and especially model theory, have even tighter relationships to computational complexity. Here, goals are different from those we pursue in programming languages: if we want to infer properties of a complexity class from properties of a certain system (which exactly characterizes it), we should keep the system as simple as possible, without emphasizing issues such as programming flexibility. Recent investigations on the correspondence between multiplicative linear logic and polynomial time go exactly in this direction [Mai04, Ter04]. 6 Complexity at Higher Types Starting from a seminal work by Constable [Con73], much effort has been devoted to extending the very notion of computational complexity to the higher-order [Coo92]. Indeed, while the notion of a feasible function (say, from natural numbers to natural numbers) is quite robust and universally accepted, the notion of a feasible functional is not completely stable yet, in particular for functionals of order strictly greater than two. This research field has produced many interesting results, from characterization results [IRK01] to type systems [DR06]. In this work, however, we stick to the usual, first-order notions of computational complexity and complexity classes, although the considered programming languages and logical systems are higher-order. 7 The Need for a Unifying Framework Although related one to the other, all the cited systems have been studied with different, often unrelated methodologies and few results are known about relative intentional expressive power. For example, it is not clear at all whether combining different systems into one language would break the correspondence with complexity classes. Similarly, it is not usually the case a system can be extended with new features preserving its quantitative properties. The author believes that implicit computational complexity needs powerful, unifying frameworks for the analysis of quantitative properties of computation. They would help to improve the understanding on existing systems. More importantly, unifying frameworks can be used themselves as a foundation for controlling the use of resources inside programming languages. This is particularly true in the area of higher-order programming languages and logical systems, where maintaining a correspondence with complexity classes is somehow more difficult. As a matter of fact, there are already some proposals on frameworks for the analysis of quantitative properties of first-order programs [Cas97, BMM04]. 8 Related Work Geometry of interaction [Gir89] has been introduced by Girard as a way of giving dynamic semantics to proofs by studying the geometry of paths in graphs representing proofs. It has been UBLCS

15 8 Related Work useful in the context of optimal reduction of functional programming languages [AG98] and abstract machines [Mac94, SP01]. A geometry of interaction model by Baillot and Pedicini [BP01] has been used to to isolate linear logic proofs that can be reduced in elementary time. It is not clear, however, whether their approach can be extended to smaller complexity classes. Context semantics [GAL92a] is a geometry of interaction model and is the subject of Part II of this thesis. Realizability interpretations have been used in connection with implicit computation complexity in several places. In Cook and Urquhart work (see [CU93]), terms of a language called PV ω realize formulas of bounded arithmetic [Bus86]. Realizability models have been used by Hofmann to prove polytime soundness of two distinct systems [Hof99a, Hof00, HS04]. A generalized realizability framework is the subject of Part III. Proposals on semantical frameworks for quantitative analysis of higher-order programs include the one by Sands [San91] and related ones [Gur91, VS03]. They are denotational models assigning abstract cost functions to programs. However, these frameworks have not been used so far in the context of implicit computational complexity. One of the reasons why this happens may be the following: in the proposed frameworks, the cost of normalization is computed from the operational semantics by assigning a constant, fixed cost to any normalization step, irrespective of the size of involved terms. This makes the task of proving the invariance [veb90] of proposed cost models more difficult (see Chapter 6 of this thesis). Another powerful framework for the analysis of complexity properties of programs and proofs consists in assigning ordinals to them [Sch99, WO02]. This way of studying the complexity of normalization has produced a number of interesting results, but few of them deal with small complexity classes. For example, the author is not aware of any characterization of polynomial time computable functions through ordinal assignments. UBLCS

16 Chapter 3 Systems and Subsystems This chapter introduces three logical and programming systems with related subsystems. We will give semantics to them in parts II and III of this thesis. We here describe their syntax and their expressive power without proving anything about the latter. 1 Multiplicative and Exponential Linear Logic We here define intuitionistic, multiplicative and exponential linear logic [Gir87] as a sequent calculus. Then, we will show how a proof-net can be associated to any sequent-calculus proof. Moreover, we will give some details on subsystems of the logic. The language of formulae is defined by the following productions: A ::= α A A A A!A α.a µα.a where α ranges over a countable set of atoms. A sequent is an object like Γ A, where Γ is a multiset of formulae and A is a formula. Notice formulae include least fixpoints µα.a; this is not usually the case in linear logic. The rules in Figure 1 define a sequent calculus for intuitionistic, multiplicative and exponential linear logic. This logic will be referred to as LL. This is slightly different from the usual IMELL. Indeed, all the systems we will talk about in this thesis are intuitionistic and do not include additive connectives. We shall use LL µ as a shorthand for this system, too. In this way, we are able to easily identify interesting fragments, such as the propositional fragment LL or the second order fragment LL. Formulae fixpoints break the cut-elimination property. However, the cut-elimination-as-computation paradigm continue to hold. Indeed, the Girard s translation A B!A B induces the following embeddings: Simply-typed lambda-calculus can be embedded into LL. Intuitionistic propositional logic can be embedded into LL [Gir87]. Intuitionistic second-order logic can be embedded into LL [Gir87]. Pure, untyped, lambda-calculus can be embedded into LL µ. Every lambda-term can be mapped to a proof with conclusion µα.!α α [Dan90]. As a consequence, these fragments of multiplicative and exponential linear logic are not interesting for implicit computational complexity. The extensional expressive power remains the same if we allow weakening to be unrestricted: Γ B Γ, A B W In this way we obtain a new system called intuitionistic affine linear logic (ALL for short). This fragment has the advantage over LL of letting the so-called additive operators to be expressible in terms of linear arrow and second order quantification. 14

17 1 Multiplicative and Exponential Linear Logic Axiom, Cut and Structural Rules A A A Γ A, A B Γ, B U Γ B Γ,!A B W Γ,!A,!A B Γ,!A B X Multiplicative Logical Rules Γ, A B Γ A B R Γ A, B C Γ,, A B C L Γ A B Γ, A B R Exponential Rules Γ, A, B C Γ, A B C L Γ B!Γ!B P! A, Γ B!A, Γ B D!!!A, Γ B!A, Γ B N! Second Order Rules Γ A α FV (Γ) Γ α.a R Γ, A{B/α} C Γ, α.a C L Least Fixpoint Rules Γ A{µα.A/α} Γ µα.a R µ Γ, A{µα.A/α} B Γ, µα.a B L µ Figure 1. A sequent calculus for intuitionistic multiplicative and exponential linear logic, LL. 1.1 Proof-nets Proof-nets [Gir87, Gir95] are graph-like representations for proofs. We here adopt a system of intuitionistic proof-nets for LL. In other words, we do not map proofs of LL µ to usual, classical, proof-nets. Proof-nets are very handy in studying the dynamics of proofs and, as a consequence, they will become our official syntax for multiplicative and exponential linear logic proofs. Let L be the set {R, L, R, L, R, L, R µ, L µ, R!, L!, W, X, D, N, P, C}. A proof-net is a graph-like structure G. It can be defined inductively as follows: a proof-net is either the graph in Figure 2(a) or one of those in Figure 3 where G, H are themselves proof-nets as in Figure 2(b). If G is a proof-net, then V G denotes the set of vertices of G, E G denotes the set of directed edges of G, α G is a labelling functions mapping every vertex in V G to an element of L and β G maps every edge in E G to a formula. Notice that each of the rules in figures 2(a) and 3 closely correspond to a rule in the sequent calculus for LL µ. Given a sequent calculus proof π, we can then build a proof-net G π corresponding to π. We should always be able to distinguish the edges which are incident to a vertex v; in particular we assume the existence of an order between them, which correspond to the clockwise order in the graphical representation of v. Nodes labelled with C (respectively, P ) mark the conclusion (respectively, the premises) of the proof-net. Notice that the rule corresponding to P! (see Figure 3) allows seeing interaction graphs as nested structures, where nodes labelled with R! and L! delimit a box. If e E G, θ G (e) denotes the vertex labelled with R! delimiting the box in which e is immediately contained (if such a box exists, otherwise θ G (e) is undefined). If v V G, θ G (v) has the same meaning. If v is a vertex with α G (v) = R!, then ρ G (v) denotes the edge de- UBLCS

18 1 Multiplicative and Exponential Linear Logic P P A 1... P A n P C 1... P C m A G H C (a) B C (b) D C Figure 2. Base cases. P... P P A 1 A n A G W B C P P... P!A A 2 A 3 A n X G A 1 B C P... A 2 G B A 1 R A 1 B P A n C P... P P P A 1 A n B 1 C 2 G C 1 L B... H D C P C n!a A 1 A 2 C P A 1... P A n G B P... C 1 H R D B D C P C m P A 1 A 2 A 2 P A 3 L A 1... G B C P A n P P... P!A 1 A 2 A n A 1 D G B C P P... P!A A 2 A n N A 1 G B C A 1!!A P... P!A 1!A n L!... L! A 1 A n G B R!!B C P... P P... P P... P A 1 A n C i 1 C 1 G C i+1c m B H D C P A 1... G B P A n P P... P α.a A 2 A n L A 1 G B P A 1... G B R µ P A n P P... P µα.a A 2 A n L µ A 1 G B R α.b C C A 1 A{E/α} µα.a C B A{µα.A/α} C A 1 A{µα.A/α} Figure 3. Inductive cases. UBLCS

19 1 Multiplicative and Exponential Linear Logic parting from v and going outside the box. If v V G, then (v) is the so-called box-depth of v, i.e. the number of boxes in which v is included. If e E G, then (e) is defined similarly. Expressions σ G (e) and σ G (v) are shorthand for ρ G (θ G (e)) and ρ G (θ G (v)), respectively. If e = (u, v) E G, and α G (u) = R!, then e is said to be a box edge. B G is the set of all box edges of G. Given a box edge e, P G (e) is the number of premises and conclusions of the box. I G is the set of all vertices v V G with α G (v) / {R!, L! }. Cut elimination is performed by graph rewriting in proof-nets. There are nine different rewriting rules S, where S C = {,,, µ,!, X, D, N, W }. We distinguish four linear rewriting rules (see Figure 4) and five exponential rewriting rules (see Figure 5). If Q C, then Q is A B A B R R A B e A B A B e A B L L A B A B... A1 An A{µα.A/α} G... Rµ µα.a e µ A{µα.A/α} B R A1 An G{C/α} Lµ α.b e B{C/α} A{µα.A/α} L B{C/α} Figure 4. Linear rewriting rules. the union of S over S Q. The relation is simply C. The notions of a normal form proof-net and of a diverging proof-net are the usual ones. A cut edge is the edge linking two nodes interacting in a cut-elimination step. In figures 4 and 5, e is always a cut edge. If S C, a cut edge linking two nodes that interacts in S is called an S-cut. Proof-nets can be thought of as a machinery implementing cut-elimination. There are some rewriting steps which are essentially global that cannot be assumed to take constant time. For example, if G X H, then the size of H can be as large as two times the size of G. However, there is a polynomial p : N 2 N such that the actual time needed to normalize a proof-net G is at most p(n, m) provided: n is an upper bound to the number of rewriting steps from G to a normal form. m is an upper bound to the size of any reduct of G. Chapter 4 shows context semantics to be a useful tool for extracting the upper bounds n and m from G. 1.2 Subsystems In this section, we will present some subsystems of LL and ALL, sketching correspondences with complexity classes. Multiplicative linear logic (MLL) is simply LL without the exponential connective!. This minimal fragment of linear logic is known for a long time [Gir87]. Its strong relations with polynomial time, however, have been discovered only recently [Mai04, Ter04]. If we add full weakening to MLL, we obtain multiplicative affine logic, MAL. This extension is completely harmless from the point of view of complexity and does not change the underlying expressive power. A characterization of elementary time can be obtained considering elementary linear logic (ELL). The grammar of formulae for ELL is the same as the one of LL, but structural and exponential rules are different (see Figure 6). In other words, ELL is obtained from LL by excluding rules N! and D! in the sequent calculus. The class of represenable functions equals the class of elementary time computable functions [Gir98, DJ03, MT03]. The expressivity is not altered switching to the affine variant EAL. UBLCS

20 1 Multiplicative and Exponential Linear Logic...!A1!An... L! L! A1 An!C1!Ci 1!A1!An!Ci+1!Cm B G R! L!... L! L!... L! L!... L! A1 An Ci 1 G Ci+1!B e......! C1 B Cm!C1!Ci 1!Ci+1!Cm L! L! L! L! L! H C1 Ci 1 B Ci+1 Cm D H R! D!D R!!D...!A1!An N... N!A1!An!A1!An!!A1!!An... L! L!... L...! L!... L! L! A1 An!A1!An A1 An!A1!An G B D D A1... D An G B N L!... L! A1 An R! G R! G!B e D B!B e N B R! B!!B!B R!!!B...!A1!An...!A1!An... L!... L! A1 An!A1 X!A1...!An X!An!A1!An L!... L! G B R! X P!... P! A1 An G P!... P! A1 An G A1 G B An... W!A1!An W... W!B e B B R!!B X!B C!!B C!!B!B e W Figure 5. Exponential rewriting rules. By altering the contraction rule of ELL, the expressive power goes down to polynomial time through soft linear logic (SLL, [Laf04]). The grammar of formulae for SLL is the same as the one of elementary linear logic, while exponential and structural rules are reported in Figure 7. As usual, the affine variant SAL has the same expressive power [BM04]. Now, observe that SLL does not capture all the polytime functions but, rather, every polytime problem. Polytime functions are characterized by light linear logic (LLL, [Gir98]). The grammar of formulae is enriched with a new production A ::= A. Exponential and structural rules are reported in Figure 8. As usual, the affine variant LAL has the same expressive power [Asp98]. Summing up, we have the following correspondences between subsystems of LL and complexity classes: UBLCS

21 2 Higher-Order Recursion Exponential and Structural Rules. Γ A!Γ!A P! Γ,!A,!A B Γ,!A B X Γ B Γ,!A B W Figure 6. Elementary Linear Logic. Exponential Rules and Multiplexing. Γ A!Γ!A P! Γ, A,..., A B Γ,!A B M Figure 7. Soft Linear Logic. MLL MAL PTIME ELL EAL FETIME SLL SAL PTIME LLL LAL FPTIME Every subsystem is placed next to the complexity class it characterizes. Here, FPTIME (respectively, FETIME) is the class of functions which can be computed in polynomial (respectively, elementary) time. PTIME is the class of problems which can be decided in polynomial time. The above correspondences can be formulated similarly to theorems 3 and 4, although every fragment has its own peculiarities (see [MT03] for an excellent survey). Moreover, notice that the correspondence does not necessarily hold for different fragments of the same logical system [DLB04]. 2 Higher-Order Recursion In this section, we will define a calculus for higher-order recursion in the style of Joachimski and Matthes [JM03]. The calculus turns out to be equivalent to Gödel s T and will be presented as a term calculus endowed with a type-assignment system in natural deduction style. Data will be represented by terms in some free algebras. As it will be shown, different free algebras do not necessarily behave in the same way from a complexity viewpoint, as opposed to what happens in computability theory. As a consequence, we cannot restrict ourselves to a canonical free algebra and need to keep all of them in our framework. A free algebra A is a couple (C A, R A ) where C A = {c A 1,...,cA k(a) } is a finite set of constructors and R A : C A N maps every constructor to its arity. A free algebra A = ({c A 1,...,cA k(a) }, R A) is a word algebra if R(c A i ) = 0 for one (and only one) i {1,..., k(a)}; R(c A j ) = 1 for every j i in {1,...,k(A)}. If A = ({c A 1,...,c A k(a) }, R A) is a word algebra, we will assume c A k(a) to be the distinguished element of C A whose arity is 0 and c A 1,...,cA k(a) 1 will denote the elements of C A whose arity is 1. Exponential Rules and Contraction. Γ, A Γ,! A P A B!A!B P! 1 A!A P! 2 Γ,!A,!A B Γ,!A B C Figure 8. Light Linear Logic. UBLCS

22 2 Higher-Order Recursion U = ({c U 1,c U 2 }, R U ) is the word algebra of unary strings. B = ({c B 1, c B 2, c B 3}, R B ) is the word algebra of binary strings. C = ({c C 1, cc 2 }, R C), where R C (c C 1 ) = 2 and R C(c C 2 ) = 0 is the free algebra of binary trees. D = ({c D 1,c D 2,c D 3 }, R D ), where R D (c D 1) = R D (c D 2) = 2 and R D (c D 3 ) = 0 is the free algebra of binary trees with binary labels. Natural numbers can be encoded by terms in U: 0 = c U 2 and n + 1 = c U 1 n for all n. In the same vein, elements of {0, 1} are in one-to-one correspondence to terms in B: ε = c B 3, and for all s {0, 1}, 0s = c B 1 s and 1s = c B 2 s. A will be a fixed, finite family {A 1,..., A n } of free algebras, where constructor sets C A1,...,C An are assumed to be pairwise disjoint. We will hereby assume U, B, C and D to be in A. K A is the maximum arity of constructors of free algebras in A, i.e. the natural number max max R A (c). A A c C A E A is the set of terms for the algebra A, while E A is the union of E A over all algebras A in A. Programs will be written in a fairly standard lambda calculus with constants (corresponding to free algebra constructors) and recursion. The latter will not be a combinator but a term former, in the style of Matthes and Joachimsky [JM03]. Moreover, we will use a term former for conditional, keeping it distinct from the one for recursion. This apparent redundancy is actually needed in presence of ramification (see, for example, [Lei93]). The language M A of terms is defined by the following productions: M ::= x c MM λx.m M {{M,...,M}} M M,..., M where c ranges over the constructors for the free algebras in A. Term formers {{,..., }} and,..., are conditional and recursion term formers, respectively. The language T A of types is defined by the following productions: A ::= A n A A where n ranges over N and A ranges over A. The indexing of base types (e.g., A 0, A 1, A 2,... ) is needed to define ramification conditions (see Section 1.2). The notation A n B is defined by induction on n as follows: A 0 B is just B, while A n+1 B is A (A n B). The level V (A) N of a type A is defined by induction on A: V (A n ) = n; V (A B) = max{v (A), V (B)}. When this does not cause ambiguity, we will denote a base type A n simply by A. The rules in Figure 9 define the assignment of types in T A to terms in M A. A type derivation π with conclusion Γ M : A will be denoted by π : Γ M : A. If there is π : Γ M : A then we will mark M as a typeable term. A type derivation π : Γ M : A is in standard form if Γ does not contain variables introduced by rule W. We will hereby assume to work with type derivations in standard form. This restriction does not affect the class of typeable terms. The recursion depth R(π) of a type derivation π : Γ M : A is the biggest number of E R A instances on any path from the root to a leaf in π. The highest tier I(π) of a type derivation π : Γ M : A is the maximum integer i such that there is an instance π 1... π n L : A i Γ, L M 1,..., M n : C of EA R inside π. Values are defined by the following productions: V ::= x λx.m T; T ::= c TT; UBLCS