# Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components

Save this PDF as:

Size: px
Start display at page:

Download "Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components"

## Transcription

1 Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components John Z. Sun Massachusetts Institute of Technology September 21, 2011

2 Outline Automata Theory Error in Automata Controlling Error Extensions 2 / 19

3 Motivation Goals Provide a framework for exploring automata Give performance guarantees from unreliable automata Relate theory to electronic or neural circuitry Historical Context First presented at Caltech by von Neumann Manuscript based on lecture notes taken by R.S. Pierce von Neumann added text to prepare for publication Five versions of the manuscript exist with different typesets, figures and content This presentation is based on the first manuscript 3 / 19

4 Automata Theory Definition A single-output automaton with time delay τ is a finite set of inputs, one output and a set of preferred subsets of inputs. The automaton stimulates its output at time t+τ if a preferred state appears at time t. Comments Automata differs from logic in that there is a time dimension We treat automata as black boxes Each input or output is allowed two states: unstimulated (0) stimulated (1) For n inputs, there exist 2 2n automata of a certain τ 4 / 19

5 Automata Theory Example Excitatory inputs Inhibitory inputs Threshold function 0, x < h; ϕ(x) = 1, x h, where h = # ex +# in. 5 / 19

6 Basic Organs Definition Two single-output automata are equivalent in the wider sense if they differ only in their time delays, not input-output behavior. Theorem Any single-output automaton is equivalent in the wider sense to a network of basic organs. There exists a unique τ such that the latter network exists iff its time delay is τ > τ Comment: Actually, any two organs above form a basis. 6 / 19

7 Single Basic Organs Scheffer Stroke (NAND) Ā = S(A,A) A B = S(S(A,B),S(A,B)) A+B = S(S(A,A),S(B,B)) Majority Organ ( best out of three ) A B = M(A,B,0) A+B = M(A,B, ) Ā can be derived from AND and OR 7 / 19

8 Examples Memory Machine: X is stimulated τ after the first time A is stimulated (i.e. memory tape) Memory Organ: X is stimulated iff A was stimulated earlier such that no stimulation of b occurred since 8 / 19

9 Error in Automata Error considerations Mechanical and electric units are subject to failure Assume: for every operation, the organ will fail to function correctly with precise probability ǫ Failures ε are assumed statistically independent of time and state of network More generally, failures can be dependent but are upper-bounded by ǫ Goal: Find the limits of the smallness of ǫ such that performance of automata can be reliable (Pr(ε) < δ) 9 / 19

10 Example: The Memory Organ Scenario Memory machine Stimulation only at time t Error probability ǫ After s cycles after t, probability organ is still stimulated is Conclusion: ρ s 1/2 as s ρ s e 2ǫs... is not so much that incorrect information will be obtained, but rather that irrelevant results will be produced. 10 / 19

11 Controlling Error 1: Multiple Machines Run m versions of network O in parallel, use majority organ to determine output values Analysis Assume η is upper bound on error of O Error at majority organ output is upper bounded by η = ǫ+(1 2ǫ)(3η 2 2η 3 ) ( Roots at 1 2,η 0 = 1 ) 1 6ǫ 1,1 η ǫ Latter two roots are only real if ǫ < 1/6 11 / 19

12 Controlling Error 1: Multiple Machines Consider successive occurences of the network (η becomes η ) If ǫ > 1/6, η 1/2 If ǫ < 1/6, η η 0 End Result: Reliable computation is possible, with necessary error level satisfying η 0 ǫ+3ǫ 2 12 / 19

13 Controlling Error 1: Multiple Machines More general argument (Sections ) Network P made up of arbitrary basic organs Each output s error probability is bounded by η 1 Use induction to reduce P into networks with smaller serial chains with maximum length denoted µ Necessary error level satisfies η 1 = 4ǫ+152ǫ 2 Procedure is impractical because new network needs 3 µ(p) as many organs as the original network P 13 / 19

14 Controlling Error 2: Multiplexing Each message is carried on a bundle of N wires For H < 1/2, stimulation of at least (1 H )N lines is interpreted as 1 For L < 1/2, stimulation of at most L N lines is 0 Everything else is malfunction Executive organ Issue: Output wires may have different values Fix: Allow for a restoring organ to ensure bundle wires have the same value 14 / 19

15 Controlling Error 2: Multiplexing Error Types Organ malfunction (with probability ǫ) Bundles not entirely stimulated or unstimulated (errors in wires) Pr(0 1) = η Pr(1 0) = ξ Goal: For a computation, find N such that probability of malfunction is η Assume: Scheffer (NAND) for executive organs, majority for restoring organ 15 / 19

16 Controlling Error 2: Multiplexing Proof Sketch Use combinatorial argument to determine probability of error propagated by wire errors Use Stirling s formula to get Gaussian approximation of error Variance increases with executive and restoring organ failures Constructive scheme 1. Design network R for function to be computed assuming error-free parts 2. Determine number of basic organs needed (denoted m) and let δ = η/m 3. Find N so that error probability of each organ is δ 4. Create multiplexed system with bundles of N wires 16 / 19

17 Controlling Error 2: Multiplexing Comments For fixed N and ǫ, δ H = δ L = 0.07 is best, according to back-of-the-envelope calculation Maximum allowable ǫ is For two practical examples, N = 20,000 is a good approximation Comparison to multiple machines Requires N as many wires and 3N as many basic organs versus exponentially more 17 / 19

18 Extensions Memory considerations Model assumes random permutations of wires in a bundle to simplify calculations This is hard to maintain when network contains feedback Multiplexing as analog computation For large N, can model bundle as an analog signal This is similar to modern logic gates Multiplexing to explain neuroscience von Neumann discusses implications in neural circuits At the time of the presentation, neural systems are the only ones dense and reliable enough to match his theory 18 / 19

19 A Final Note In the introduction, von Neumann writes: Our present treatment of error is unsatisfactory and ad hoc. It is the author s conviction, voiced over many years, that error should be treated by thermodynamical methods and be the subject of a thermodynamical theory, as information has been by the work of L. Szilard and C.E. Shannon. 19 / 19

### CS 256: Neural Computation Lecture Notes

CS 56: Neural Computation Lecture Notes Chapter : McCulloch-Pitts Neural Networks A logical calculus of the ideas immanent in nervous activity M. Minsky s adaptation (Computation: Finite and Infinite Machines,

### From Greek philosophers to circuits: An introduction to boolean logic. COS 116, Spring 2011 Sanjeev Arora

From Greek philosophers to circuits: An introduction to boolean logic. COS 116, Spring 2011 Sanjeev Arora Midterm One week from today in class Mar 10 Covers lectures, labs, homework, readings to date You

### Boolean Algebra, Gates and Circuits

Boolean Algebra, Gates and Circuits Kasper Brink November 21, 2017 (Images taken from Tanenbaum, Structured Computer Organization, Fifth Edition, (c) 2006 Pearson Education, Inc.) Outline Last week: Von

### MIT Manufacturing Systems Analysis Lectures 6 9: Flow Lines

2.852 Manufacturing Systems Analysis 1/165 Copyright 2010 c Stanley B. Gershwin. MIT 2.852 Manufacturing Systems Analysis Lectures 6 9: Flow Lines Models That Can Be Analyzed Exactly Stanley B. Gershwin

### Simple Neural Nets for Pattern Classification: McCulloch-Pitts Threshold Logic CS 5870

Simple Neural Nets for Pattern Classification: McCulloch-Pitts Threshold Logic CS 5870 Jugal Kalita University of Colorado Colorado Springs Fall 2014 Logic Gates and Boolean Algebra Logic gates are used

### Conditional Statements

Conditional Statements Lecture 3 Section 2.2 Robb T. Koether Hampden-Sydney College Fri, Jan 17, 2014 Robb T. Koether (Hampden-Sydney College) Conditional Statements Fri, Jan 17, 2014 1 / 26 1 Conditional

### CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010

CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010 So far our notion of realistic computation has been completely deterministic: The Turing Machine

### Computer Science. 19. Combinational Circuits. Computer Science COMPUTER SCIENCE. Section 6.1.

COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I I : A L G O R I T H M S, M A C H I N E S, a n d T H E O R Y Computer Science Computer Science An Interdisciplinary Approach Section 6.1 ROBERT SEDGEWICK

### Motivation. Evolution has rediscovered several times multicellularity as a way to build complex living systems

Cellular Systems 1 Motivation Evolution has rediscovered several times multicellularity as a way to build complex living systems Multicellular systems are composed by many copies of a unique fundamental

### The State Explosion Problem

The State Explosion Problem Martin Kot August 16, 2003 1 Introduction One from main approaches to checking correctness of a concurrent system are state space methods. They are suitable for automatic analysis

### 1 Randomized complexity

80240233: Complexity of Computation Lecture 6 ITCS, Tsinghua Univesity, Fall 2007 23 October 2007 Instructor: Elad Verbin Notes by: Zhang Zhiqiang and Yu Wei 1 Randomized complexity So far our notion of

### Quantum Computation. Michael A. Nielsen. University of Queensland

Quantum Computation Michael A. Nielsen University of Queensland Goals: 1. To eplain the quantum circuit model of computation. 2. To eplain Deutsch s algorithm. 3. To eplain an alternate model of quantum

### Compilers. Lexical analysis. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Lecture 3 Lexical analysis Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Big picture Source code Front End IR Back End Machine code Errors Front end responsibilities Check

### Lecture 6: Time-Dependent Behaviour of Digital Circuits

Lecture 6: Time-Dependent Behaviour of Digital Circuits Two rather different quasi-physical models of an inverter gate were discussed in the previous lecture. The first one was a simple delay model. This

### Boolean Algebra. Digital Logic Appendix A. Boolean Algebra Other operations. Boolean Algebra. Postulates, Identities in Boolean Algebra

Digital Logic Appendix A Gates Combinatorial Circuits Sequential Circuits George Boole ideas 1854 Claude Shannon, apply to circuit design, 1938 (piirisuunnittelu) Describe digital circuitry function programming

### EGC221: Digital Logic Lab

Division of Engineering Programs EGC221: Digital Logic Lab Experiment #1 Basic Logic Gate Simulation Student s Name: Student s Name: Reg. no.: Reg. no.: Semester: Fall 2016 Date: 07 September 2016 Assessment:

### Lecture 4: Perceptrons and Multilayer Perceptrons

Lecture 4: Perceptrons and Multilayer Perceptrons Cognitive Systems II - Machine Learning SS 2005 Part I: Basic Approaches of Concept Learning Perceptrons, Artificial Neuronal Networks Lecture 4: Perceptrons

### Systems I: Computer Organization and Architecture

Systems I: Computer Organization and Architecture Lecture 6 - Combinational Logic Introduction A combinational circuit consists of input variables, logic gates, and output variables. The logic gates accept

### Error Threshold for Individual Faulty Gates Using Probabilistic Transfer Matrix (PTM)

Available online at www.sciencedirect.com ScienceDirect AASRI Procedia 9 (2014 ) 138 145 2014 AASRI Conference on Circuits and Signal Processing (CSP 2014) Error Threshold for Individual Faulty Gates Using

### 1 The Low-Degree Testing Assumption

Advanced Complexity Theory Spring 2016 Lecture 17: PCP with Polylogarithmic Queries and Sum Check Prof. Dana Moshkovitz Scribes: Dana Moshkovitz & Michael Forbes Scribe Date: Fall 2010 In this lecture

### Section 3.3: Discrete-Event Simulation Examples

Section 33: Discrete-Event Simulation Examples Discrete-Event Simulation: A First Course c 2006 Pearson Ed, Inc 0-13-142917-5 Discrete-Event Simulation: A First Course Section 33: Discrete-Event Simulation

### Boolean Algebra. Digital Logic Appendix A. Postulates, Identities in Boolean Algebra How can I manipulate expressions?

Digital Logic Appendix A Gates Combinatorial Circuits Sequential Circuits Other operations NAND A NAND B = NOT ( A ANDB) = AB NOR A NOR B = NOT ( A ORB) = A + B Truth tables What is the result of the operation

### LOGIC CIRCUITS. Basic Experiment and Design of Electronics

Basic Experiment and Design of Electronics LOGIC CIRCUITS Ho Kyung Kim, Ph.D. hokyung@pusan.ac.kr School of Mechanical Engineering Pusan National University Outline Combinational logic circuits Output

### Chapter 9. Estimating circuit speed. 9.1 Counting gate delays

Chapter 9 Estimating circuit speed 9.1 Counting gate delays The simplest method for estimating the speed of a VLSI circuit is to count the number of VLSI logic gates that the input signals must propagate

### Stochastic Computational Models for Accurate Reliability Evaluation of Logic Circuits

Stochastic Computational Models for Accurate Evaluation of Logic Circuits Hao Chen and Jie Han Department of Electrical and Computer Engineering, University of Alberta Edmonton, Alberta, Canada T6G 2V4

### 7. Combinational Circuits

7. Combinational Circuits Jacob Abraham Department of Electrical and Computer Engineering The University of Texas at Austin VLSI Design Fall 2017 September 25, 2017 ECE Department, University of Texas

### Homing and Synchronizing Sequences

Homing and Synchronizing Sequences Sven Sandberg Information Technology Department Uppsala University Sweden 1 Outline 1. Motivations 2. Definitions and Examples 3. Algorithms (a) Current State Uncertainty

### Computer Science. Questions for discussion Part II. Computer Science COMPUTER SCIENCE. Section 4.2.

COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I I : A L G O R I T H M S, T H E O R Y, A N D M A C H I N E S Computer Science Computer Science An Interdisciplinary Approach Section 4.2 ROBERT SEDGEWICK

### Turing Machines Part II

Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

### Lecture 9 and 10: Malicious Security - GMW Compiler and Cut and Choose, OT Extension

CS 294 Secure Computation February 16 and 18, 2016 Lecture 9 and 10: Malicious Security - GMW Compiler and Cut and Choose, OT Extension Instructor: Sanjam Garg Scribe: Alex Irpan 1 Overview Garbled circuits

### Laboratory Exercise #8 Introduction to Sequential Logic

Laboratory Exercise #8 Introduction to Sequential Logic ECEN 248: Introduction to Digital Design Department of Electrical and Computer Engineering Texas A&M University 2 Laboratory Exercise #8 1 Introduction

### Lecture 6: Logical Effort

Lecture 6: Logical Effort Outline Logical Effort Delay in a Logic Gate Multistage Logic Networks Choosing the Best Number of Stages Example Summary Introduction Chip designers face a bewildering array

### From Sequential Circuits to Real Computers

1 / 36 From Sequential Circuits to Real Computers Lecturer: Guillaume Beslon Original Author: Lionel Morel Computer Science and Information Technologies - INSA Lyon Fall 2017 2 / 36 Introduction What we

### Lecture 4. Finite Automata and Safe State Machines (SSM) Daniel Kästner AbsInt GmbH 2012

Lecture 4 Finite Automata and Safe State Machines (SSM) Daniel Kästner AbsInt GmbH 2012 Initialization Analysis 2 Is this node well initialized? node init1() returns (out: int) let out = 1 + pre( 1 ->

### On the Effect of Analog Noise in Discrete-Time Analog Computations

On the Effect of Analog Noise in Discrete-Time Analog Computations Wolfgang Maass Institute for Theoretical Computer Science Technische Universitat Graz* Pekka Orponen Department of Mathematics University

### Machine Learning Lecture 5

Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory

### Finally the Weakest Failure Detector for Non-Blocking Atomic Commit

Finally the Weakest Failure Detector for Non-Blocking Atomic Commit Rachid Guerraoui Petr Kouznetsov Distributed Programming Laboratory EPFL Abstract Recent papers [7, 9] define the weakest failure detector

### On optimal stopping of autoregressive sequences

On optimal stopping of autoregressive sequences Sören Christensen (joint work with A. Irle, A. Novikov) Mathematisches Seminar, CAU Kiel September 24, 2010 Sören Christensen (CAU Kiel) Optimal stopping

### Probabilistic Models in Theoretical Neuroscience

Probabilistic Models in Theoretical Neuroscience visible unit Boltzmann machine semi-restricted Boltzmann machine restricted Boltzmann machine hidden unit Neural models of probabilistic sampling: introduction

### Lecture 14: State Tables, Diagrams, Latches, and Flip Flop

EE210: Switching Systems Lecture 14: State Tables, Diagrams, Latches, and Flip Flop Prof. YingLi Tian Nov. 6, 2017 Department of Electrical Engineering The City College of New York The City University

### Equivalence of DFAs and NFAs

CS 172: Computability and Complexity Equivalence of DFAs and NFAs It s a tie! DFA NFA Sanjit A. Seshia EECS, UC Berkeley Acknowledgments: L.von Ahn, L. Blum, M. Blum What we ll do today Prove that DFAs

### A Review of the E 3 Algorithm: Near-Optimal Reinforcement Learning in Polynomial Time

A Review of the E 3 Algorithm: Near-Optimal Reinforcement Learning in Polynomial Time April 16, 2016 Abstract In this exposition we study the E 3 algorithm proposed by Kearns and Singh for reinforcement

### One Lesson of Information Theory

Institut für One Lesson of Information Theory Prof. Dr.-Ing. Volker Kühn Institute of Communications Engineering University of Rostock, Germany Email: volker.kuehn@uni-rostock.de http://www.int.uni-rostock.de/

### HOPFIELD neural networks (HNNs) are a class of nonlinear

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 52, NO. 4, APRIL 2005 213 Stochastic Noise Process Enhancement of Hopfield Neural Networks Vladimir Pavlović, Member, IEEE, Dan Schonfeld,

### The L Machines are very high-level, in two senses:

What is a Computer? State of the machine. CMPSCI 630: Programming Languages An Abstract Machine for Control Spring 2009 (with thanks to Robert Harper) Internal registers, memory, etc. Initial and final

### Approximate Message Passing Algorithms

November 4, 2017 Outline AMP (Donoho et al., 2009, 2010a) Motivations Derivations from a message-passing perspective Limitations Extensions Generalized Approximate Message Passing (GAMP) (Rangan, 2011)

### Static CMOS Circuits. Example 1

Static CMOS Circuits Conventional (ratio-less) static CMOS Covered so far Ratio-ed logic (depletion load, pseudo nmos) Pass transistor logic ECE 261 Krish Chakrabarty 1 Example 1 module mux(input s, d0,

### Simple Iteration, cont d

Jim Lambers MAT 772 Fall Semester 2010-11 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Simple Iteration, cont d In general, nonlinear equations cannot be solved in a finite sequence

### Lecture 8: Sequential Networks and Finite State Machines

Lecture 8: Sequential Networks and Finite State Machines CSE 140: Components and Design Techniques for Digital Systems Spring 2014 CK Cheng, Diba Mirza Dept. of Computer Science and Engineering University

### DM7404 Hex Inverting Gates

DM7404 Hex Inverting Gates General Description This device contains six independent gates each of which performs the logic INVERT function. Ordering Code: August 1986 Revised February 2000 DM7404 Hex Inverting

### Majority Logic Decoding under. Data-Dependent Logic Gate Failures

Majority Logic Decoding under 1 Data-Dependent Logic Gate Failures Srdan Brkic, Student Member, IEEE, Predrag Ivaniš, Member, IEEE, and Bane Vasić, Fellow, IEEE arxiv:1507.07155v1 [cs.it] 26 Jul 2015 Abstract

### A call-by-name lambda-calculus machine

A call-by-name lambda-calculus machine Jean-Louis Krivine University Paris VII, C.N.R.S. 2 place Jussieu 75251 Paris cedex 05 (krivine@pps.jussieu.fr) Introduction We present, in this paper, a particularly

### An Introduction to Quantum Computation and Quantum Information

An to and Graduate Group in Applied Math University of California, Davis March 13, 009 A bit of history Benioff 198 : First paper published mentioning quantum computing Feynman 198 : Use a quantum computer

### Lecture 3: Semantics of Propositional Logic

Lecture 3: Semantics of Propositional Logic 1 Semantics of Propositional Logic Every language has two aspects: syntax and semantics. While syntax deals with the form or structure of the language, it is

### Bayesian Machine Learning

Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September

### A Note on the Reduction of Two-Way Automata to One-Way Automata

A Note on the Reduction of Two-Way Automata to One-Way Automata Moshe Y. Vardi IBM Almaden Research Center Abstract We describe a new elementary reduction of two-way automata to one-way automata. The reduction

### Turing Machines Part II

Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture

### Löwenheim-Skolem Theorems, Countable Approximations, and L ω. David W. Kueker (Lecture Notes, Fall 2007)

Löwenheim-Skolem Theorems, Countable Approximations, and L ω 0. Introduction David W. Kueker (Lecture Notes, Fall 2007) In its simplest form the Löwenheim-Skolem Theorem for L ω1 ω states that if σ L ω1

### Uniformly Uniformly-ergodic Markov chains and BSDEs

Uniformly Uniformly-ergodic Markov chains and BSDEs Samuel N. Cohen Mathematical Institute, University of Oxford (Based on joint work with Ying Hu, Robert Elliott, Lukas Szpruch) Centre Henri Lebesgue,

### Cellular Automata: Tutorial

Cellular Automata: Tutorial Jarkko Kari Department of Mathematics, University of Turku, Finland TUCS(Turku Centre for Computer Science), Turku, Finland Cellular Automata: examples A Cellular Automaton

### Sample Test Paper - I

Scheme G Sample Test Paper - I Course Name : Computer Engineering Group Marks : 25 Hours: 1 Hrs. Q.1) Attempt any THREE: 09 Marks a) Define i) Propagation delay ii) Fan-in iii) Fan-out b) Convert the following:

### Localized Excitations in Networks of Spiking Neurons

Localized Excitations in Networks of Spiking Neurons Hecke Schrobsdorff Bernstein Center for Computational Neuroscience Göttingen Max Planck Institute for Dynamics and Self-Organization Seminar: Irreversible

### CIRCUITS AND ELECTRONICS. The Digital Abstraction

6.002 CIRCUITS AND ELECTRONICS The Digital Abstraction Review Discretize matter by agreeing to observe the lumped matter discipline Lumped Circuit Abstraction Analysis tool kit: KVL/KCL, node method, superposition,

### Undecidable Problems and Reducibility

University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.

### Notes for Lecture 3... x 4

Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

### What is thermal equilibrium and how do we get there?

arxiv:1507.06479 and more What is thermal equilibrium and how do we get there? Hal Tasaki QMath 13, Oct. 9, 2016, Atlanta 40 C 20 C 30 C 30 C about the talk Foundation of equilibrium statistical mechanics

### LECTURE 28. Analyzing digital computation at a very low level! The Latch Pipelined Datapath Control Signals Concept of State

Today LECTURE 28 Analyzing digital computation at a very low level! The Latch Pipelined Datapath Control Signals Concept of State Time permitting, RC circuits (where we intentionally put in resistance

### IN THIS paper we investigate the diagnosability of stochastic

476 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL 50, NO 4, APRIL 2005 Diagnosability of Stochastic Discrete-Event Systems David Thorsley and Demosthenis Teneketzis, Fellow, IEEE Abstract We investigate

### Data Analysis and Monte Carlo Methods

Lecturer: Allen Caldwell, Max Planck Institute for Physics & TUM Recitation Instructor: Oleksander (Alex) Volynets, MPP & TUM General Information: - Lectures will be held in English, Mondays 16-18:00 -

### The uniform uncertainty principle and compressed sensing Harmonic analysis and related topics, Seville December 5, 2008

The uniform uncertainty principle and compressed sensing Harmonic analysis and related topics, Seville December 5, 2008 Emmanuel Candés (Caltech), Terence Tao (UCLA) 1 Uncertainty principles A basic principle

### Mutual Information, Synergy and Some Curious Phenomena for Simple Channels

Mutual Information, Synergy and Some Curious henomena for Simple Channels I. Kontoyiannis Div of Applied Mathematics & Dpt of Computer Science Brown University rovidence, RI 9, USA Email: yiannis@dam.brown.edu

### DM74LS02 Quad 2-Input NOR Gate

Quad 2-Input NOR Gate General Description This device contains four independent gates each of which performs the logic NOR function. Ordering Code: May 1986 Revised March 2000 Order Number Package Number

### ROM-BASED COMPUTATION: QUANTUM VERSUS CLASSICAL

arxiv:quant-ph/0109016v2 2 Jul 2002 ROM-BASED COMPUTATION: QUANTUM VERSUS CLASSICAL B. C. Travaglione, M. A. Nielsen Centre for Quantum Computer Technology, University of Queensland St Lucia, Queensland,

### NCU EE -- DSP VLSI Design. Tsung-Han Tsai 1

NCU EE -- DSP VLSI Design. Tsung-Han Tsai 1 Multi-processor vs. Multi-computer architecture µp vs. DSP RISC vs. DSP RISC Reduced-instruction-set Register-to-register operation Higher throughput by using

### Randomness and non-uniformity

Randomness and non-uniformity JASS 2006 Course 1: Proofs and Computers Felix Weninger TU München April 2006 Outline Randomized computation 1 Randomized computation 2 Computation with advice Non-uniform

### Part 1: Digital Logic and Gates. Analog vs. Digital waveforms. The digital advantage. In real life...

Part 1: Digital Logic and Gates Analog vs Digital waveforms An analog signal assumes a continuous range of values: v(t) ANALOG A digital signal assumes discrete (isolated, separate) values Usually there

### A strongly polynomial algorithm for linear systems having a binary solution

A strongly polynomial algorithm for linear systems having a binary solution Sergei Chubanov Institute of Information Systems at the University of Siegen, Germany e-mail: sergei.chubanov@uni-siegen.de 7th

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### (Classical) Information Theory II: Source coding

(Classical) Information Theory II: Source coding Sibasish Ghosh The Institute of Mathematical Sciences CIT Campus, Taramani, Chennai 600 113, India. p. 1 Abstract The information content of a random variable

### Models. Models of Computation, Turing Machines, and the Limits of Turing Computation. Effective Calculability. Motivation for Models of Computation

Turing Computation /0/ Models of Computation, Turing Machines, and the Limits of Turing Computation Bruce MacLennan Models A model is a tool intended to address a class of questions about some domain of

### Notes on Complexity Theory Last updated: November, Lecture 10

Notes on Complexity Theory Last updated: November, 2015 Lecture 10 Notes by Jonathan Katz, lightly edited by Dov Gordon. 1 Randomized Time Complexity 1.1 How Large is BPP? We know that P ZPP = RP corp

### Hybrid Control and Switched Systems. Lecture #1 Hybrid systems are everywhere: Examples

Hybrid Control and Switched Systems Lecture #1 Hybrid systems are everywhere: Examples João P. Hespanha University of California at Santa Barbara Summary Examples of hybrid systems 1. Bouncing ball 2.

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### Chapter 3. Chapter 3 :: Topics. Introduction. Sequential Circuits

Chapter 3 Chapter 3 :: Topics igital esign and Computer Architecture, 2 nd Edition avid Money Harris and Sarah L. Harris Introduction Latches and Flip Flops Synchronous Logic esign Finite State Machines

### Universal Memcomputing Machines

1 Universal Memcomputing Machines Fabio L. Traversa, Massimiliano Di Ventra arxiv:1405.0931v2 [cs.ne] 12 Nov 2014 Abstract We introduce the notion of universal memcomputing machines (UMMs): a class of

### (Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann

(Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for

### 2DI90 Probability & Statistics. 2DI90 Chapter 4 of MR

2DI90 Probability & Statistics 2DI90 Chapter 4 of MR Recap - Random Variables ( 2.8 MR)! Example: X is the random variable corresponding to the temperature of the room at time t. x is the measured temperature

### 14:332:231 DIGITAL LOGIC DESIGN. Organizational Matters (1)

4:332:23 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical & Computer Engineering Fall 23 Organizational Matters () Instructor: Ivan MARSIC Office: CoRE Building, room 7 Email: marsic@ece.rutgers.edu

### Lecture 2: January 18

CS271 Randomness & Computation Spring 2018 Instructor: Alistair Sinclair Lecture 2: January 18 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They

### CprE 281: Digital Logic

CprE 28: Digital Logic Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs/classes/ Decoders and Encoders CprE 28: Digital Logic Iowa State University, Ames, IA Copyright Alexander Stoytchev

### Theory of Computation p.1/?? Theory of Computation p.2/?? Unknown: Implicitly a Boolean variable: true if a word is

Abstraction of Problems Data: abstracted as a word in a given alphabet. Σ: alphabet, a finite, non-empty set of symbols. Σ : all the words of finite length built up using Σ: Conditions: abstracted as a

### Normal Forms of Propositional Logic

Normal Forms of Propositional Logic Bow-Yaw Wang Institute of Information Science Academia Sinica, Taiwan September 12, 2017 Bow-Yaw Wang (Academia Sinica) Normal Forms of Propositional Logic September

### Chapter 15. System Reliability Concepts and Methods. William Q. Meeker and Luis A. Escobar Iowa State University and Louisiana State University

Chapter 15 System Reliability Concepts and Methods William Q. Meeker and Luis A. Escobar Iowa State University and Louisiana State University Copyright 1998-2008 W. Q. Meeker and L. A. Escobar. Based on

### Memory, Latches, & Registers

Memory, Latches, & Registers 1) Structured Logic Arrays 2) Memory Arrays 3) Transparent Latches 4) How to save a few bucks at toll booths 5) Edge-triggered Registers L13 Memory 1 General Table Lookup Synthesis

### EDUCATIONAL PEARL. Proof-directed debugging revisited for a first-order version

JFP 16 (6): 663 670, 2006. c 2006 Cambridge University Press doi:10.1017/s0956796806006149 First published online 14 September 2006 Printed in the United Kingdom 663 EDUCATIONAL PEARL Proof-directed debugging

### Reification of Boolean Logic

526 U1180 neural networks 1 Chapter 1 Reification of Boolean Logic The modern era of neural networks began with the pioneer work of McCulloch and Pitts (1943). McCulloch was a psychiatrist and neuroanatomist;

### DM74LS08 Quad 2-Input AND Gates

DM74LS08 Quad 2-Input AND Gates General Description This device contains four independent gates each of which performs the logic AND function. Ordering Code: August 1986 Revised March 2000 Order Number

### Regular Languages and Finite Automata

Regular Languages and Finite Automata Theorem: Every regular language is accepted by some finite automaton. Proof: We proceed by induction on the (length of/structure of) the description of the regular