lecture 3
Readings until now This week Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters: 1 and 2 Gleick, J. [2011]. The Information: A History, a Theory, a Flood. Random House. Chapter 8. Optional McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133. Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters 11, and 12. Presentations Freeth, Tony. 2014. Eclipse Prediction on the Ancient Greek Astronomical Calculating Machine Known as the Antikythera Mechanism. PloS One 9 (7): e103275. Plank, Nicholas Wasserman, M., X.H.T. Zeng, and L.A.N. Amaral [2015]. Crossevaluation of metrics to estimate the significance of creative works. PNAS 112 (5) 1281-1286. Boothby, Clara Lecture Notes The Nature of Information Available http://canvas.iu.edu and listed at http://informatics.indiana.edu/rocha/academics/i501 Also check out Links and notes at http://sciber.blogspot.com/
Charles Babbage (1791 1871) and Ada Lovelace (1815-1852) The external tape as a general principle (system) of universal computing Analytical engine Separated memory store from a central processing unit (or mill ) Cogs not just numbers variables Programmable instructions on punched cards Inspired by the Jacquard Loom Ada Lovelace: the science of operations Set of (recursive) rules for producing Bernoulli numbers (a program) Separation of variable and operational (data) cards would punch out cards for later use the Engine eating its own tail. (Babbage) distinction between numbers that mean things and numbers that do things.
key contributions (most relevant to biocomplexity) The chemical basis of morphogenesis Turing, A. M. Phil. Trans. R. Soc. Lond. B 237, 37 72 (1952). Reaction-diffusion systems Computing machinery and intelligence Turing, A. M. Mind 49, 433 460 (1950). The Turing Test On computable numbers with an application to the Entscheidungsproblem Turing, A. M. Proc. Lond. Math. Soc. s2 42, 230 265 (1936 37). Turing machine, universal computation, decision problem Brenner, Sydney. [2012]. Life s code script. Nature 482 (7386): 461-461. Alan Turing (1912-1954)
Computing with numbers It is always possible to encode any symbol string into a sequence of integers A sequence of integers can be mapped into a single natural number Gödelization (prime factorization) Computer scientists can concentrate on functions that take a single number as input and output Computation as a mapping of numbers to other numbers Models of Computation How to construct number mappings Turing machine, general recursive functions, λ-calculus computation as number crunching
A fundamental principle of computation Turing s tape On computable numbers with an application to the Entscheidungsproblem Turing, A. M. Proc. Lond. Math. Soc. s2 42, 230 265 (1936 37). Turing machine, universal computation, decision problem Machine s state is controlled by a program, while data for program is on limitless external tape every machine can be described as a number that can be stored on the tape for another machine Including a Universal machine distinction between numbers that mean things (data) and numbers that do things (program) The fundamental, indivisible unit of information is the bit. The fundamental, indivisible unit of digital computation is the transformation of a bit between its two possible forms of existence: as [memory] or as [code]. George Dyson, 2012.
At every discrete time instance the machine is in a single state Program is a state transition table state Read symbol Next state Write symbol Tape move 0 4 1 - left 1-0 1 right
At every discrete time instance the machine is in a single state Program is a state transition table state Read symbol Next state Write symbol Tape move 0 4 1 - left 1-0 1 right
some facts computation Process of rewriting strings in a formal system according to a program of rules Operations and states are syntactic Symbols follow syntactical rules Rate of computation is irrelevant Program determines result, not speed of machine Physical implementation is irrelevant for result Computer Physical device that can reliably execute/approximate a formal computation Errors always exist Design aims to make rate and dynamics irrelevant [ ] essential elements in the machine are of a binary [ ] nature. Those whose state is determined by their history and are timestable are memory elements. Elements of which the state is determined essentially by the existing amplitude of a voltage or signal are called gates. Bigelow et al, 1947
John Von Neumann (1903-1957) Turing machines beyond the decision problem Words coding the orders are handled in the memory just like numbers --- distinction between numbers that mean things and numbers that do things. realizing the power of Turing s tape physical (electronic) computers emphasized the importance of the storedprogram concept (the external tape) EDVAC allows machine to modify its own program von Neumann architecture: The functional separation of storage from the processing unit. programs can exist as data (two roles) Converts tape to fixed-address memory (random-access memory) Let the whole outside world consist of a long paper tape. John von Neumann, 1948
John Von Neumann (1903-1957) Turing machines beyond the decision problem Words coding the orders are handled in the memory just like numbers --- distinction between numbers that mean things and numbers that do things. realizing the power of Turing s tape physical (electronic) computers emphasized the importance of the storedprogram concept (the external tape) EDVAC allows machine to modify its own program von Neumann architecture: The functional separation of storage from the processing Since Babbage s machine was not electrical, and since all digital computers are in a sense equivalent, we see that this use of electricity cannot be of theoretical importance. The feature of using electricity is thus seen to be only a very unit. superficial similarity. (Alan Turing) programs can exist as data (two roles) Converts tape to fixed-address memory (random-access memory) Let the whole outside world consist of a long paper tape. John von Neumann, 1948
design principles of computation Babbage/Lovelace, Turing s tape, and roles of information distinction between numbers that mean things and numbers that do things.
Nature.com; ANDY POTTS; TURING FAMILY
1942 meeting other pre-cybernetics developments The Cerebral Inhibition Meeting New York City, May 1942 Organized by Frank Freemont-Smith of the Josiah Macy Jr. Foundation From the human sciences Lawrence Frank, Margaret Mead and Gregory Bateson From the sciences Warren McCulloch and Arturo Rosenblueth Result Huge excitement about Rosenblueth s presentation of concepts from Norbert Wiener and Julien Bigelow Homeostasis, purposeful action (goal-direction), aiming A new paradigm of interdisciplinary research? Goal-directed actions Controversial: explaining actions in terms of future events, violating cause and effect Teleological mechanisms Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: Statedetermined systems The mathematics were accessible
McCulloch & Pitts Memory can be maintained in circular networks of binary switches McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133. A Turing machine program could be implemented in a finite network of binary neuron/switches Neurons as basic computing unit of the brain Circularity is essential for memory (closed loops to sustain memory) Brain (mental?) function as computing Others at Macy Meeting emphasized other aspects of brain activity Chemical concentrations and field effects (not digital) Ralph Gerard and Fredrik Bremmer
Cybernetics was born post-war science: the Josiah Macy Jr. Foundation Meetings The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences March 1946 (10 meetings between 1946 and 1953) Interdisciplinary Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let s look at them with a war-time team culture Participants John Von Neumann, Leonard Savage, Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc. Key concepts Homeostasis, Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-determined systems The mathematics were finally accessible
Informatics Cybernetics was born post-war science: the Josiah Macy Jr. Foundation Meetings The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences Interdisciplinary Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let s look at them with a war-time team culture Participants March 1946 (10 meetings between 1946 and 1953) John Von Neumann, Leonard Savage, Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc. Key concepts Homeostasis, Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-determined systems The mathematics were finally accessible
post-war science Synthetic approach Engineering-inspired Supremacy of mechanism Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology All can be axiomatized and computed Mculloch&Pitts work was major influence Macy Conferences: 1946-53 A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5:115-133 (1943). A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present) cybernetics Warren S. McCulloch Margaret Mead Claude Shannon
at the Macy meetings other key concepts Norbert Wiener and Arturo Rosenblueth Goal-directed behavior and negative feedback (control) Homeostasis and circular causality In machines and biology Automata Theory (Von Neumann) Communication and Information The fundamental idea is the message, even though the message may not be sent by man and the fundamental element of the message is the decision (Norbert Wiener) Shannon s Information and Wiener s Communication Theory Natural semiotics (McCulloch and others later get into Peircean Semiotics) functional equivalence of systems (general systems) Bio-inspired mathematics and engineering and computing/mechanism-inspired biology and social science
at the Macy meetings Gregory Bateson and Margaret Mead Homeostasis and circular causality in society Transvestite ceremony to diffuse aggressive action in Iatmul culture Learning and evolution Can a computer learn to learn? A new organizing principle for the social sciences (control and communication) As much as evolution was for Biology Lawrence Frank The new interdisciplinary concepts needed a new kind of language Higher generality than what is used in single topic disciplines A call for a science of systems Yehoshua Bar-Hillel Optimism of a new (cybernetics and information) age A new synthesis [ ] was destined to open new vistas on everything human to help solve many of the disturbing open problems concerning man and humanity. other key concepts
Turing as cybernetician British Cybernetics The Ratio Club (starting in1949) British cybernetics meetings William Ross Ashby, W. Grey Walter, Alan Turing. etc computation or the faculty of mind which calculates, plans and reasons Also following Wiener s use of Machina ratiocinatrix in Cybernetics (1948), following Leibniz calculus ratiocinator
Turing as cybernetician British Cybernetics The Ratio Club (starting in1949) British cybernetics meetings William Ross Ashby, W. Grey Walter, Alan Turing. etc computation or the faculty of mind which calculates, plans and reasons Also following Wiener s use of Machina ratiocinatrix in Cybernetics (1948), following Leibniz calculus ratiocinator
controlling information to achieve life-like behavior trial and error algorithm information as reduction of uncertainty in the presence of alternatives (combinatorics) lifelike behavior trial and error to learn path from many alternatives adapts to new situations how is learning achieved? Correct choices, information gained from reduced uncertainty, must be stored in memory memory of information as a design principle of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays Brain as (switching) machine Shannon s mouse
Readings (available in Canvas) next class Week 4 Lecture Prokopenko, Mikhail, Fabio Boschetti, and Alex J. Ryan. [2009]. "An information-theoretic primer on complexity, self-organization, and emergence. Complexity 15(1): 11-28 Presentations & Discussion Barabasi and Albert (1999) Emergence of Scaling in Random Networks, Science 296 (5439). Erkol, Sirag Hofman, Jake M., Amit Sharma, and Duncan J. Watts. "Prediction and explanation in social systems." Science 355.6324 (2017): 486-488. Kaminski, Patrick Lecture Notes The Nature of Information Optional Aleksander, I. [2002]. Understanding Information Bit by Bit. In: It must be beautiful : great equations of modern science. G. Farmelo (Ed.), Granta, London.