lecture 3 Informatics luis rocha 2017 I501 introduction to informatics INDIANA UNIVERSITY

Size: px

Start display at page:

Download "lecture 3 Informatics luis rocha 2017 I501 introduction to informatics INDIANA UNIVERSITY"

Kelly Gray
5 years ago
Views:

1 lecture 3

Readings until now This week Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters: 1 and 2 Gleick, J. [2011]. The Information: A History, a Theory, a Flood. Random House. Chapter 8.

MIT Press. Chapters 11, and 12. Presentations Freeth, Tony. 2014. Eclipse Prediction on the Ancient Greek Astronomical Calculating Machine Known as the Antikythera Mechanism. PloS One 9 (7): e103275.

2 Readings until now This week Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters: 1 and 2 Gleick, J. [2011]. The Information: A History, a Theory, a Flood. Random House. Chapter 8. Optional McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5: Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters 11, and 12. Presentations Freeth, Tony Eclipse Prediction on the Ancient Greek Astronomical Calculating Machine Known as the Antikythera Mechanism. PloS One 9 (7): e Plank, Nicholas Wasserman, M., X.H.T. Zeng, and L.A.N. Amaral [2015]. Crossevaluation of metrics to estimate the significance of creative works. PNAS 112 (5) Boothby, Clara Lecture Notes The Nature of Information Available and listed at Also check out Links and notes at

Charles Babbage (1791 1871) and Ada Lovelace

principle (system) of universal computing

from a central processing unit (or mill )

instructions on punched cards Inspired by

of operations Set of (recursive) rules for

3 Charles Babbage ( ) and Ada Lovelace ( ) The external tape as a general principle (system) of universal computing Analytical engine Separated memory store from a central processing unit (or mill ) Cogs not just numbers variables Programmable instructions on punched cards Inspired by the Jacquard Loom Ada Lovelace: the science of operations Set of (recursive) rules for producing Bernoulli numbers (a program) Separation of variable and operational (data) cards would punch out cards for later use the Engine eating its own tail. (Babbage) distinction between numbers that mean things and numbers that do things.

The Turing Test On computable numbers with an application to the Entscheidungsproblem Turing, A. M. Proc. Lond. Math. Soc.

4 key contributions (most relevant to biocomplexity) The chemical basis of morphogenesis Turing, A. M. Phil. Trans. R. Soc. Lond. B 237, (1952). Reaction-diffusion systems Computing machinery and intelligence Turing, A. M. Mind 49, (1950). The Turing Test On computable numbers with an application to the Entscheidungsproblem Turing, A. M. Proc. Lond. Math. Soc. s2 42, ( ). Turing machine, universal computation, decision problem Brenner, Sydney. [2012]. Life s code script. Nature 482 (7386): Alan Turing ( )

Computing with numbers It is always possible to encode any symbol string into a sequence of integers A sequence of integers can be mapped into a single natural number Gödelization (prime

5 Computing with numbers It is always possible to encode any symbol string into a sequence of integers A sequence of integers can be mapped into a single natural number Gödelization (prime factorization) Computer scientists can concentrate on functions that take a single number as input and output Computation as a mapping of numbers to other numbers Models of Computation How to construct number mappings Turing machine, general recursive functions, λ-calculus computation as number crunching

A fundamental principle of computation Turing s tape On computable numbers with an application to the Entscheidungsproblem Turing,

Turing machine, universal computation, decision problem Machine s state is controlled by a program, while data for program is on

Universal machine distinction between numbers that mean things (data) and numbers that do things (program) The fundamental,

6 A fundamental principle of computation Turing s tape On computable numbers with an application to the Entscheidungsproblem Turing, A. M. Proc. Lond. Math. Soc. s2 42, ( ). Turing machine, universal computation, decision problem Machine s state is controlled by a program, while data for program is on limitless external tape every machine can be described as a number that can be stored on the tape for another machine Including a Universal machine distinction between numbers that mean things (data) and numbers that do things (program) The fundamental, indivisible unit of information is the bit. The fundamental, indivisible unit of digital computation is the transformation of a bit between its two possible forms of existence: as [memory] or as [code]. George Dyson, 2012.

7 At every discrete time instance the machine is in a single state Program is a state transition table state Read symbol Next state Write symbol Tape move left right

8 At every discrete time instance the machine is in a single state Program is a state transition table state Read symbol Next state Write symbol Tape move left right

some facts computation Process of rewriting strings in a formal system according to a program of rules Operations and states are syntactic Symbols follow syntactical rules Rate of computation is

9 some facts computation Process of rewriting strings in a formal system according to a program of rules Operations and states are syntactic Symbols follow syntactical rules Rate of computation is irrelevant Program determines result, not speed of machine Physical implementation is irrelevant for result Computer Physical device that can reliably execute/approximate a formal computation Errors always exist Design aims to make rate and dynamics irrelevant [ ] essential elements in the machine are of a binary [ ] nature. Those whose state is determined by their history and are timestable are memory elements. Elements of which the state is determined essentially by the existing amplitude of a voltage or signal are called gates. Bigelow et al, 1947

John Von Neumann (1903-1957) Turing machines beyond the decision problem Words coding the orders are handled in the memory just like numbers --- distinction between numbers that mean things and

10 John Von Neumann ( ) Turing machines beyond the decision problem Words coding the orders are handled in the memory just like numbers --- distinction between numbers that mean things and numbers that do things. realizing the power of Turing s tape physical (electronic) computers emphasized the importance of the storedprogram concept (the external tape) EDVAC allows machine to modify its own program von Neumann architecture: The functional separation of storage from the processing unit. programs can exist as data (two roles) Converts tape to fixed-address memory (random-access memory) Let the whole outside world consist of a long paper tape. John von Neumann, 1948

11 John Von Neumann ( ) Turing machines beyond the decision problem Words coding the orders are handled in the memory just like numbers --- distinction between numbers that mean things and numbers that do things. realizing the power of Turing s tape physical (electronic) computers emphasized the importance of the storedprogram concept (the external tape) EDVAC allows machine to modify its own program von Neumann architecture: The functional separation of storage from the processing Since Babbage s machine was not electrical, and since all digital computers are in a sense equivalent, we see that this use of electricity cannot be of theoretical importance. The feature of using electricity is thus seen to be only a very unit. superficial similarity. (Alan Turing) programs can exist as data (two roles) Converts tape to fixed-address memory (random-access memory) Let the whole outside world consist of a long paper tape. John von Neumann, 1948

12 design principles of computation Babbage/Lovelace, Turing s tape, and roles of information distinction between numbers that mean things and numbers that do things.

13 Nature.com; ANDY POTTS; TURING FAMILY

1942 meeting other pre-cybernetics developments The Cerebral Inhibition Meeting New York City, May 1942 Organized by Frank Freemont-Smith

Foundation From the human sciences Lawrence Frank, Margaret Mead and Gregory Bateson From the sciences Warren McCulloch and Arturo

purposeful action (goal-direction), aiming A new paradigm of interdisciplinary research?

14 1942 meeting other pre-cybernetics developments The Cerebral Inhibition Meeting New York City, May 1942 Organized by Frank Freemont-Smith of the Josiah Macy Jr. Foundation From the human sciences Lawrence Frank, Margaret Mead and Gregory Bateson From the sciences Warren McCulloch and Arturo Rosenblueth Result Huge excitement about Rosenblueth s presentation of concepts from Norbert Wiener and Julien Bigelow Homeostasis, purposeful action (goal-direction), aiming A new paradigm of interdisciplinary research? Goal-directed actions Controversial: explaining actions in terms of future events, violating cause and effect Teleological mechanisms Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: Statedetermined systems The mathematics were accessible

A Turing machine program could be implemented in a finite network of binary neuron/switches Neurons as basic computing unit of the brain Circularity is

15 McCulloch & Pitts Memory can be maintained in circular networks of binary switches McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5: A Turing machine program could be implemented in a finite network of binary neuron/switches Neurons as basic computing unit of the brain Circularity is essential for memory (closed loops to sustain memory) Brain (mental?) function as computing Others at Macy Meeting emphasized other aspects of brain activity Chemical concentrations and field effects (not digital) Ralph Gerard and Fredrik Bremmer

16 Cybernetics was born post-war science: the Josiah Macy Jr. Foundation Meetings The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences March 1946 (10 meetings between 1946 and 1953) Interdisciplinary Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let s look at them with a war-time team culture Participants John Von Neumann, Leonard Savage, Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc. Key concepts Homeostasis, Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-determined systems The mathematics were finally accessible

17 Informatics Cybernetics was born post-war science: the Josiah Macy Jr. Foundation Meetings The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences Interdisciplinary Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let s look at them with a war-time team culture Participants March 1946 (10 meetings between 1946 and 1953) John Von Neumann, Leonard Savage, Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc. Key concepts Homeostasis, Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-determined systems The mathematics were finally accessible

post-war science Synthetic approach Engineering-inspired Supremacy of mechanism Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology All can be axiomatized and

18 post-war science Synthetic approach Engineering-inspired Supremacy of mechanism Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology All can be axiomatized and computed Mculloch&Pitts work was major influence Macy Conferences: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5: (1943). A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present) cybernetics Warren S. McCulloch Margaret Mead Claude Shannon

at the Macy meetings other key concepts Norbert Wiener and Arturo Rosenblueth Goal-directed behavior and negative feedback (control) Homeostasis and circular causality In

fundamental element of the message is the decision (Norbert Wiener) Shannon s Information and Wiener s Communication Theory Natural semiotics (McCulloch and others later get

19 at the Macy meetings other key concepts Norbert Wiener and Arturo Rosenblueth Goal-directed behavior and negative feedback (control) Homeostasis and circular causality In machines and biology Automata Theory (Von Neumann) Communication and Information The fundamental idea is the message, even though the message may not be sent by man and the fundamental element of the message is the decision (Norbert Wiener) Shannon s Information and Wiener s Communication Theory Natural semiotics (McCulloch and others later get into Peircean Semiotics) functional equivalence of systems (general systems) Bio-inspired mathematics and engineering and computing/mechanism-inspired biology and social science

at the Macy meetings Gregory Bateson and Margaret Mead Homeostasis and circular causality in society Transvestite ceremony to diffuse aggressive action in Iatmul culture Learning and

A new organizing principle for the social sciences (control and communication) As much as evolution was for Biology Lawrence Frank The new interdisciplinary concepts needed a new kind of

20 at the Macy meetings Gregory Bateson and Margaret Mead Homeostasis and circular causality in society Transvestite ceremony to diffuse aggressive action in Iatmul culture Learning and evolution Can a computer learn to learn? A new organizing principle for the social sciences (control and communication) As much as evolution was for Biology Lawrence Frank The new interdisciplinary concepts needed a new kind of language Higher generality than what is used in single topic disciplines A call for a science of systems Yehoshua Bar-Hillel Optimism of a new (cybernetics and information) age A new synthesis [ ] was destined to open new vistas on everything human to help solve many of the disturbing open problems concerning man and humanity. other key concepts

21 Turing as cybernetician British Cybernetics The Ratio Club (starting in1949) British cybernetics meetings William Ross Ashby, W. Grey Walter, Alan Turing. etc computation or the faculty of mind which calculates, plans and reasons Also following Wiener s use of Machina ratiocinatrix in Cybernetics (1948), following Leibniz calculus ratiocinator

William Ross Ashby, W. Grey Walter, Alan Turing.

plans and reasons Also following Wiener s use of Machina

22 Turing as cybernetician British Cybernetics The Ratio Club (starting in1949) British cybernetics meetings William Ross Ashby, W. Grey Walter, Alan Turing. etc computation or the faculty of mind which calculates, plans and reasons Also following Wiener s use of Machina ratiocinatrix in Cybernetics (1948), following Leibniz calculus ratiocinator

controlling information to achieve life-like behavior trial and error

alternatives (combinatorics) lifelike behavior trial and error to

stored in memory memory of information as a design principle of

23 controlling information to achieve life-like behavior trial and error algorithm information as reduction of uncertainty in the presence of alternatives (combinatorics) lifelike behavior trial and error to learn path from many alternatives adapts to new situations how is learning achieved? Correct choices, information gained from reduced uncertainty, must be stored in memory memory of information as a design principle of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays Brain as (switching) machine Shannon s mouse

24 Readings (available in Canvas) next class Week 4 Lecture Prokopenko, Mikhail, Fabio Boschetti, and Alex J. Ryan. [2009]. "An information-theoretic primer on complexity, self-organization, and emergence. Complexity 15(1): Presentations & Discussion Barabasi and Albert (1999) Emergence of Scaling in Random Networks, Science 296 (5439). Erkol, Sirag Hofman, Jake M., Amit Sharma, and Duncan J. Watts. "Prediction and explanation in social systems." Science (2017): Kaminski, Patrick Lecture Notes The Nature of Information Optional Aleksander, I. [2002]. Understanding Information Bit by Bit. In: It must be beautiful : great equations of modern science. G. Farmelo (Ed.), Granta, London.

complex networks and systems i609: advanced PhD seminar I Informatics luis rocha 2019 INDIANA UNIVERSITY

complex networks and systems Informatics luis rocha 2019 i609: advanced PhD seminar I luis m. rocha school of informatics, computing, & engineering indiana university, bloomington, usa and instituto gulbenkian