# Welcome to Comp 411! 2) Course Objectives. 1) Course Mechanics. 3) Information. I thought this course was called Computer Organization

Save this PDF as:
Size: px
Start display at page:

Download "Welcome to Comp 411! 2) Course Objectives. 1) Course Mechanics. 3) Information. I thought this course was called Computer Organization"

## Transcription

1 Welcome to Comp 4! I thought this course was called Computer Organization David Macaulay ) Course Mechanics 2) Course Objectives 3) Information L - Introduction

2 Meet the Crew Lectures: Leonard McMillan (SN-258) Office Hours T 2-3 TA: TBA (We ll find out next Monday) Book: Patterson & Hennessy Computer Organization & Design 3 rd Edition, ISBN: (However, you won t need it for the next couple of weeks) L - Introduction 2

3 Course Mechanics Grading: Best 8 of problem sets 4% 2 Quizzes 3% Final Exam 3% Problem sets will be distributed on Wednesdays and are due back on the next Wednesday *lecture* (before it begins). Usually you will have one week to complete each set. Late problem sets will not be accepted, but the lowest two problem-set scores will be dropped. I will attempt to make Lecture Notes, Problem Sets, and other course materials available on the web Before Class on the day they are given. L - Introduction 3

4 Comp 4: Course Website Announcements, corrections, etc. On-line copies of all handouts L - Introduction 4

5 Goal : Demystify Computers Strangely, most people (even some computer scientists) are afraid of computers. We are only afraid of things we do not understand! I do not fear computers. I fear the lack of them. - Isaac Asimov (92-992) Fear is the main source of superstition, and one of the main sources of cruelty. To conquer fear is the beginning of wisdom. - Bertrand Russell (872 97) L - Introduction 5

6 Goal 2: Power of Abstraction Define a function, develop a robust implementation, and then put a box around it. Abstraction enables us to create unfathomable systems (including computers). Why do we need ABSTRACTION Imagine a billion ---,,, L - Introduction 6

7 The key to building systems with >G components Personal Computer: Hardware & Software Circuit Board: 8 / system -2G devices Integrated Circuit: 8-6 / PCB.25M-6M devices Module: 8-6 / IC K devices MOSFET Scheme for representing information Gate: 2-6 / Cell 8 devices Cell: K-K / Module 6-64 devices L - Introduction 7

8 What do we See in a Computer? Structure hierarchical design: Interfaces limited complexity at each level reusable building blocks Key elements of system engineering; typically outlive the technologies they interface Isolate technologies, allow evolution Major abstraction mechanism What makes a good system design? Bang for the buck : minimal mechanism, maximal function reliable in a wide range of environments accommodates future technical improvements Wait! I think I see a bug in the DIV logic. Got that one off the web. Sure hope it works. L - Introduction 8

9 Computational Structures What are the fundamental elements of computation? Can we define computation independent of implementation or the substrate that it built upon)? L - Introduction 9

10 Our Plan of Attack Understand how things work, by alternating between low-level (bottom-up) and high level (top-down) concepts Encapsulate our understanding using appropriate abstractions Study organizational principles: hierarchy, interfaces, APIs. Roll up our sleeves and design at each level of hierarchy Learn engineering tricks - from a historical perspective - using systematic design approaches - diagnose, fix, and avoid bugs L - Introduction

11 What is Computation? Computation is about processing information - Transforming information from one form to another - Deriving new information from old - Finding information associated with a given input - Computation describes the motion of information through time - Communication describes the motion of information through space L - Introduction

12 What is Information? information, n. Knowledge communicated or received concerning a particular fact or circumstance. Problem sets, 2 quizzes, and a final! Tarheels won! Are you sure? It is football season A Computer Scientist s Definition: Information resolves uncertainty. Information is simply that which cannot be predicted. The less predictable a message is, the more information it conveys! L - Introduction 2

13 Real-World Information Why do unexpected messages get allocated the biggest headlines? because they carry the most information. L - Introduction 3

14 What Does A Computer Process? Toasters processes bread and bagels Blenders processes smoothies and margaritas What does a computer process? 2 allowable answers: Information Bits How does information relate to bits? L - Introduction 4

15 Quantifying Information (Claude Shannon, 948) Suppose you re faced with N equally probable choices, and I give you a fact that narrows it down to M choices. Then you ve been given: log 2 (N/M) bits of information Information is measured in bits (binary digits) = number of / s required to encode choice(s) Examples: information in one coin flip: log 2 (2/) = bit roll of a single die: log 2 (6/) = ~2.6 bits outcome of a Football game: bit (well, actually, they won may convey more information than they lost ) L - Introduction 5

16 Example: Sum of 2 dice i 2 = log 2 (36/) = 5.7 bits i 3 = log 2 (36/2) = 4.7 bits i 4 = log 2 (36/3) = bits i 5 = log 2 (36/4) = 3.7 bits i 6 = log 2 (36/5) = bits i 7 = log 2 (36/6) = bits i 8 = log 2 (36/5) = bits i 9 = log 2 (36/4) = 3.7 bits i = log 2 (36/3) = bits i = log 2 (36/2) = 4.7 bits i 2 = log 2 (36/) = 5.7 bits The average information provided by the sum of 2 dice: Entropy i 2 M i ave = log 2 ( N ) = p i log 2 ( p N M i ) = i i = 2 i bits L - Introduction 6

17 Show Me the Bits! Can the sum of two dice REALLY be represented using bits? If so, how? The fact is, the average information content is a strict *lower-bound* on how small of a representation that we can achieve. In practice, it is difficult to reach this bound. But, we can come very close. L - Introduction 7

18 Variable-Length Encoding Of course we can use differing numbers of bits to represent each item of data This is particularly useful if all items are *not* equally likely Equally likely items lead to fixed length encodings: Ex: Encode a particular roll of 5? {(,4), (2,3), (3,2), (4,)} which are equally likely if we use fair dice Entropy = bits 4 p( roll 5)log ( ( 5)) iroll = 2 p rolliroll = = log 2( ) = 4 4 i = i= (,4), (2,3), (3,2), (4,) Back to the original problem. Let s use this encoding: L - Introduction 8

19 Variable-Length Encoding Taking a closer look Unlikely rolls are encoded using more bits More likely rolls use fewer bits Decoding Example Stream: L - Introduction 9

20 Huffman Coding A simple *greedy* algorithm for approximating an entropy efficient encoding. Find the 2 items with the smallest probabilities 2. Join them into a new *meta* item whose probability is the sum 3. Remove the two items and insert the new meta item 4. Repeat from step until there is only one item 36/36 Huffman decoding tree 5/36 2/36 7/36 /36 8/36 /36 9 4/36 5 4/36 4/36 4 3/36 5/36 7 6/36 8 5/36 6 5/36 2/36 3 2/36 3/36 2/36 2 /36 2 /36 L - Introduction 2

21 Converting Tree to Encoding Once the *tree* is constructed, label its edges consistently and follow the paths from the largest *meta* item to each of the real item to find the encoding /36 36/36 Huffman decoding tree 2/36 9 4/36 8/36 5 4/36 2/36 4/36 7/36 3 2/36 4 3/36 3/36 5/36 2 /36 /36 2/36 7 6/36 2 /36 8 5/36 /36 6 5/36 L - Introduction 2

22 Encoding Efficiency How does this encoding strategy compare to the information content of the roll? b ave = b ave = Pretty close. Recall that the lower bound was bits. However, an efficient encoding (as defined by having an average code size close to the information content) is not always what we want! L - Introduction 22

23 Encoding Considerations - Encoding schemes that attempt to match the information content of a data stream remove redundancy. They are data compression techniques. - However, sometimes our goal in encoding information is increase redundancy, rather than remove it. Why? - Make the information easier to manipulate (fixed-sized encodings) - Make the data stream resilient to noise (error detecting and correcting codes) -Data compression allows us to store our entire music collection in a pocketable device -Data redundancy enables us to store that *same* information *reliably* on a hard drive L - Introduction 23

24 Error detection using parity Sometimes we add extra redundancy so that we can detect errors. For instance, this encoding detects any single-bit error: ex: 4 6* 7 : 4 9* 7 2: 4 * 7 3: 4 2* 5 4: Same bitstream w/4 possible interpretations if we allow for only one error There s something peculiar about those codings L - Introduction 24

25 Property : Parity The sum of the bits in each symbol is even. (this is how errors are detected) 2- = = 4 3- = = 6 4- = = 2 5- = = 2 6- = = 2 7- = = 8- = = 2 9- = = 2 - = = 2 - = = 6 2- = = 6 How much information is in the last bit? L - Introduction 25

26 Property 2: Separation Each encoding differs from all others by at least two bits in their overlapping parts xx xx xx xx xxxx xx xx xx xx xx 3 xx xx xx xxxx xx xx xx xx xx 4 xx xx xx xx xx xxxx xx xx 5 xx xx xx xxxx xx xx xx 6 xx xxxx xx xx xx xx 7 xx xx xx xxxx xxxx 8 xx xx xx xx 9 xx xx xx xx xx This difference is called xx the Hamming distance A Hamming distance of one-bit is needed to uniquely identify an encoding L - Introduction 26

27 Error correcting codes We can actually correct -bit errors in encodings separated by a Hamming distance of three. This is possible because the sets of bit patterns located a Hamming distance of from our encodings are distinct. However, attempting error correction with such a small separation is dangerous. Suppose, we have a 2-bit error. Our error correction scheme will then misinterpret the encoding. Misinterpretations also occurred when we had 2-bit errors in our -bit-error-detection (parity) schemes. A safe -bit error correction scheme would correct all -bit errors and detect all 2-bit errors. What Hamming distance is needed between encodings to accomplish this? This is just a voting scheme L - Introduction 27

28 An alternate error correcting code We can generalize the notion of parity in order to construct error correcting codes. Instead of computing a single parity bit for an entire encoding we can allow multiple parity bits over different subsets of bits. Consider the following technique for encoding 25 bits. This approach is easy to implement, but it is not optimal in terms of the number of bits used. -bit errors will cause both a row and column parity error, uniquely identifying the errant bit for correction. An extra parity bit allows us to detect -bit errors in the parity bits. Many 2-bit errors can also be corrected. However, 2-bit errors involving the same row or column are undetectable. L - Introduction 28

29 Summary Information resolves uncertainty Choices equally probable: N choices down to M log 2 (N/M) bits of information Choices not equally probable: choice i with probability p i log 2 (/p i ) bits of information average number of bits = Σp i log 2 (/p i ) use variable-length encodings Next time: Technology and Abstraction What makes a computer tick L - Introduction 29

### Lecture 1: Shannon s Theorem

Lecture 1: Shannon s Theorem Lecturer: Travis Gagie January 13th, 2015 Welcome to Data Compression! I m Travis and I ll be your instructor this week. If you haven t registered yet, don t worry, we ll work

### Lecture 11: Information theory THURSDAY, FEBRUARY 21, 2019

Lecture 11: Information theory DANIEL WELLER THURSDAY, FEBRUARY 21, 2019 Agenda Information and probability Entropy and coding Mutual information and capacity Both images contain the same fraction of black

### 1. Basics of Information

1. Basics of Information 6.004x Computation Structures Part 1 Digital Circuits Copyright 2015 MIT EECS 6.004 Computation Structures L1: Basics of Information, Slide #1 What is Information? Information,

### 6.02 Fall 2011 Lecture #9

6.02 Fall 2011 Lecture #9 Claude E. Shannon Mutual information Channel capacity Transmission at rates up to channel capacity, and with asymptotically zero error 6.02 Fall 2011 Lecture 9, Slide #1 First

### CprE 281: Digital Logic

CprE 28: Digital Logic Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs/classes/ Code Converters CprE 28: Digital Logic Iowa State University, Ames, IA Copyright Alexander Stoytchev HW

### An introduction to basic information theory. Hampus Wessman

An introduction to basic information theory Hampus Wessman Abstract We give a short and simple introduction to basic information theory, by stripping away all the non-essentials. Theoretical bounds on

### 6.02 Fall 2012 Lecture #1

6.02 Fall 2012 Lecture #1 Digital vs. analog communication The birth of modern digital communication Information and entropy Codes, Huffman coding 6.02 Fall 2012 Lecture 1, Slide #1 6.02 Fall 2012 Lecture

### Discrete Mathematics and Probability Theory Fall 2013 Vazirani Note 12. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Fall 203 Vazirani Note 2 Random Variables: Distribution and Expectation We will now return once again to the question of how many heads in a typical sequence

### Probability (Devore Chapter Two)

Probability (Devore Chapter Two) 1016-345-01: Probability and Statistics for Engineers Fall 2012 Contents 0 Administrata 2 0.1 Outline....................................... 3 1 Axiomatic Probability 3

### Information and Entropy. Professor Kevin Gold

Information and Entropy Professor Kevin Gold What s Information? Informally, when I communicate a message to you, that s information. Your grade is 100/100 Information can be encoded as a signal. Words

### Roll No. :... Invigilator's Signature :.. CS/B.TECH(ECE)/SEM-7/EC-703/ CODING & INFORMATION THEORY. Time Allotted : 3 Hours Full Marks : 70

Name : Roll No. :.... Invigilator's Signature :.. CS/B.TECH(ECE)/SEM-7/EC-703/2011-12 2011 CODING & INFORMATION THEORY Time Allotted : 3 Hours Full Marks : 70 The figures in the margin indicate full marks

### Discrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 15. Random Variables: Distributions, Independence, and Expectations

EECS 70 Discrete Mathematics and Probability Theory Fall 204 Anant Sahai Note 5 Random Variables: Distributions, Independence, and Expectations In the last note, we saw how useful it is to have a way of

### Information in Biology

Lecture 3: Information in Biology Tsvi Tlusty, tsvi@unist.ac.kr Living information is carried by molecular channels Living systems I. Self-replicating information processors Environment II. III. Evolve

### 20. Combinational Circuits

Combinational circuits Q. What is a combinational circuit? A. A digital circuit (all signals are or ) with no feedback (no loops). analog circuit: signals vary continuously sequential circuit: loops allowed

### Discrete Mathematics and Probability Theory Fall 2012 Vazirani Note 14. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Fall 202 Vazirani Note 4 Random Variables: Distribution and Expectation Random Variables Question: The homeworks of 20 students are collected in, randomly

### CPSC 121 Midterm 1 Tuesday, October 11th, 2011

CPSC 121 Midterm 1 Tuesday, October 11th, 2011 [1] 1. Do you want tutorial attendance to be mandatory for you? If you answer yes, then 1% of tuts attended your course grade will be calculated with min(100%,

### Information in Biology

Information in Biology CRI - Centre de Recherches Interdisciplinaires, Paris May 2012 Information processing is an essential part of Life. Thinking about it in quantitative terms may is useful. 1 Living

### 1 Ex. 1 Verify that the function H(p 1,..., p n ) = k p k log 2 p k satisfies all 8 axioms on H.

Problem sheet Ex. Verify that the function H(p,..., p n ) = k p k log p k satisfies all 8 axioms on H. Ex. (Not to be handed in). looking at the notes). List as many of the 8 axioms as you can, (without

### CpE358/CS381. Switching Theory and Logical Design. Summer

Switching Theory and Logical Design - Class Schedule Monday Tuesday Wednesday Thursday Friday May 7 8 9 - Class 2 - Class 2 2 24 - Class 3 25 26 - Class 4 27 28 Quiz Commencement 3 June 2 - Class 5 3 -

### Information & Correlation

Information & Correlation Jilles Vreeken 11 June 2014 (TADA) Questions of the day What is information? How can we measure correlation? and what do talking drums have to do with this? Bits and Pieces What

### Decision Trees. Nicholas Ruozzi University of Texas at Dallas. Based on the slides of Vibhav Gogate and David Sontag

Decision Trees Nicholas Ruozzi University of Texas at Dallas Based on the slides of Vibhav Gogate and David Sontag Supervised Learning Input: labelled training data i.e., data plus desired output Assumption:

### Course Review. Kin 304W Week 14: April 9, 2013

Course Review Kin 304W Week 14: April 9, 2013 1 Today s Outline Format of Kin 304W Final Exam Course Review Hand back marked Project Part II 2 Kin 304W Final Exam Saturday, Thursday, April 18, 3:30-6:30

### Computer Science. 19. Combinational Circuits. Computer Science COMPUTER SCIENCE. Section 6.1.

COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I I : A L G O R I T H M S, M A C H I N E S, a n d T H E O R Y Computer Science Computer Science An Interdisciplinary Approach Section 6.1 ROBERT SEDGEWICK

### Applications. Smartphone, tablet, game controller, antilock brakes, microprocessor, Wires

COMPUTER SCIENCE Combinational circuits Q. What is a combinational circuit? A. A digital circuit (all signals are or ) with no feedback (no loops). analog circuit: signals vary continuously sequential

### Lecture 7: DecisionTrees

Lecture 7: DecisionTrees What are decision trees? Brief interlude on information theory Decision tree construction Overfitting avoidance Regression trees COMP-652, Lecture 7 - September 28, 2009 1 Recall:

### We are here. Assembly Language. Processors Arithmetic Logic Units. Finite State Machines. Circuits Gates. Transistors

CSC258 Week 3 1 Logistics If you cannot login to MarkUs, email me your UTORID and name. Check lab marks on MarkUs, if it s recorded wrong, contact Larry within a week after the lab. Quiz 1 average: 86%

### Class Website:

ECE 20B, Winter 2003 Introduction to Electrical Engineering, II LECTURE NOTES #5 Instructor: Andrew B. Kahng (lecture) Email: abk@ece.ucsd.edu Telephone: 858-822-4884 office, 858-353-0550 cell Office:

### Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 16. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Spring 206 Rao and Walrand Note 6 Random Variables: Distribution and Expectation Example: Coin Flips Recall our setup of a probabilistic experiment as

### Decision Trees. CSC411/2515: Machine Learning and Data Mining, Winter 2018 Luke Zettlemoyer, Carlos Guestrin, and Andrew Moore

Decision Trees Claude Monet, The Mulberry Tree Slides from Pedro Domingos, CSC411/2515: Machine Learning and Data Mining, Winter 2018 Luke Zettlemoyer, Carlos Guestrin, and Andrew Moore Michael Guerzhoy

### Shannon's Theory of Communication

Shannon's Theory of Communication An operational introduction 5 September 2014, Introduction to Information Systems Giovanni Sileno g.sileno@uva.nl Leibniz Center for Law University of Amsterdam Fundamental

### CprE 281: Digital Logic

CprE 28: Digital Logic Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs/classes/ Decoders and Encoders CprE 28: Digital Logic Iowa State University, Ames, IA Copyright Alexander Stoytchev

### Entropies & Information Theory

Entropies & Information Theory LECTURE I Nilanjana Datta University of Cambridge,U.K. See lecture notes on: http://www.qi.damtp.cam.ac.uk/node/223 Quantum Information Theory Born out of Classical Information

### Example: sending one bit of information across noisy channel. Effects of the noise: flip the bit with probability p.

Lecture 20 Page 1 Lecture 20 Quantum error correction Classical error correction Modern computers: failure rate is below one error in 10 17 operations Data transmission and storage (file transfers, cell

### 1. Counting. Chris Piech and Mehran Sahami. Oct 2017

. Counting Chris Piech and Mehran Sahami Oct 07 Introduction Although you may have thought you had a pretty good grasp on the notion of counting at the age of three, it turns out that you had to wait until

### EE 229B ERROR CONTROL CODING Spring 2005

EE 229B ERROR CONTROL CODING Spring 2005 Solutions for Homework 1 1. Is there room? Prove or disprove : There is a (12,7) binary linear code with d min = 5. If there were a (12,7) binary linear code with

### Sequential Logic (3.1 and is a long difficult section you really should read!)

EECS 270, Fall 2014, Lecture 6 Page 1 of 8 Sequential Logic (3.1 and 3.2. 3.2 is a long difficult section you really should read!) One thing we have carefully avoided so far is feedback all of our signals

### Computer Science 324 Computer Architecture Mount Holyoke College Fall Topic Notes: Digital Logic

Computer Science 324 Computer Architecture Mount Holyoke College Fall 2007 Topic Notes: Digital Logic Our goal for the next few weeks is to paint a a reasonably complete picture of how we can go from transistor

### CS6375: Machine Learning Gautam Kunapuli. Decision Trees

Gautam Kunapuli Example: Restaurant Recommendation Example: Develop a model to recommend restaurants to users depending on their past dining experiences. Here, the features are cost (x ) and the user s

### Entropy as a measure of surprise

Entropy as a measure of surprise Lecture 5: Sam Roweis September 26, 25 What does information do? It removes uncertainty. Information Conveyed = Uncertainty Removed = Surprise Yielded. How should we quantify

### E40M. Binary Numbers. M. Horowitz, J. Plummer, R. Howe 1

E40M Binary Numbers M. Horowitz, J. Plummer, R. Howe 1 Reading Chapter 5 in the reader A&L 5.6 M. Horowitz, J. Plummer, R. Howe 2 Useless Box Lab Project #2 Adding a computer to the Useless Box alows us

### Probability (Devore Chapter Two)

Probability (Devore Chapter Two) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 0 Preliminaries 3 0.1 Motivation..................................... 3 0.2 Administrata...................................

### Digital communication system. Shannon s separation principle

Digital communication system Representation of the source signal by a stream of (binary) symbols Adaptation to the properties of the transmission channel information source source coder channel coder modulation

### Computer Science. 20. Combinational Circuits. Computer Science COMPUTER SCIENCE. Section

COMPUTER SCIENCE S E D G E W I C K / W A Y N E Computer Science 20. Combinational Circuits Computer Science An Interdisciplinary Approach Section 6.1 ROBERT SEDGEWICK K E V I N WAY N E http://introcs.cs.princeton.edu

### Designing Information Devices and Systems I Fall 2018 Lecture Notes Note Introduction to Linear Algebra the EECS Way

EECS 16A Designing Information Devices and Systems I Fall 018 Lecture Notes Note 1 1.1 Introduction to Linear Algebra the EECS Way In this note, we will teach the basics of linear algebra and relate it

### UNIT I INFORMATION THEORY. I k log 2

UNIT I INFORMATION THEORY Claude Shannon 1916-2001 Creator of Information Theory, lays the foundation for implementing logic in digital circuits as part of his Masters Thesis! (1939) and published a paper

### Digital Communications III (ECE 154C) Introduction to Coding and Information Theory

Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I

### Massachusetts Institute of Technology

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science Department of Mechanical Engineering 6.050J/2.0J Information and Entropy Spring 2005 Issued: March 7, 2005

### L14. 1 Lecture 14: Crash Course in Probability. July 7, Overview and Objectives. 1.2 Part 1: Probability

L14 July 7, 2017 1 Lecture 14: Crash Course in Probability CSCI 1360E: Foundations for Informatics and Analytics 1.1 Overview and Objectives To wrap up the fundamental concepts of core data science, as

### Lecture 1. Introduction

Lecture 1. Introduction What is the course about? Logistics Questionnaire Dr. Yao Xie, ECE587, Information Theory, Duke University What is information? Dr. Yao Xie, ECE587, Information Theory, Duke University

### Software Testing Lecture 2

Software Testing Lecture 2 Justin Pearson September 25, 2014 1 / 1 Test Driven Development Test driven development (TDD) is a way of programming where all your development is driven by tests. Write tests

### Massachusetts Institute of Technology. Solution to Problem 1: The Registrar s Worst Nightmare

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science Department of Mechanical Engineering 6.050J/2.0J Information and Entropy Spring 2006 Issued: March 4, 2006

### S C F F F T T F T T S C B F F F F F T F T F F T T T F F T F T T T F T T T

EECS 270, Winter 2017, Lecture 1 Page 1 of 6 Use pencil! Say we live in the rather black and white world where things (variables) are either true (T) or false (F). So if S is Mark is going to the Store

### Logic. Combinational. inputs. outputs. the result. system can

Digital Electronics Combinational Logic Functions Digital logic circuits can be classified as either combinational or sequential circuits. A combinational circuit is one where the output at any time depends

### ! Where are we on course map? ! What we did in lab last week. " How it relates to this week. ! Compression. " What is it, examples, classifications

Lecture #3 Compression! Where are we on course map?! What we did in lab last week " How it relates to this week! Compression " What is it, examples, classifications " Probability based compression # Huffman

### Memory, Latches, & Registers

Memory, Latches, & Registers 1) Structured Logic Arrays 2) Memory Arrays 3) Transparent Latches 4) How to save a few bucks at toll booths 5) Edge-triggered Registers L13 Memory 1 General Table Lookup Synthesis

### Physical Layer and Coding

Physical Layer and Coding Muriel Médard Professor EECS Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media:

### Counting. 1 Sum Rule. Example 1. Lecture Notes #1 Sept 24, Chris Piech CS 109

1 Chris Piech CS 109 Counting Lecture Notes #1 Sept 24, 2018 Based on a handout by Mehran Sahami with examples by Peter Norvig Although you may have thought you had a pretty good grasp on the notion of

### Math 1313 Experiments, Events and Sample Spaces

Math 1313 Experiments, Events and Sample Spaces At the end of this recording, you should be able to define and use the basic terminology used in defining experiments. Terminology The next main topic in

### Practice Second Midterm Exam I

CS103 Handout 33 Fall 2018 November 2, 2018 Practice Second Midterm Exam I This exam is closed-book and closed-computer. You may have a double-sided, 8.5 11 sheet of notes with you when you take this exam.

### Information Theory. Week 4 Compressing streams. Iain Murray,

Information Theory http://www.inf.ed.ac.uk/teaching/courses/it/ Week 4 Compressing streams Iain Murray, 2014 School of Informatics, University of Edinburgh Jensen s inequality For convex functions: E[f(x)]

### What s the Deal? MULTIPLICATION. Time to multiply

What s the Deal? MULTIPLICATION Time to multiply Multiplying two numbers requires a multiply Luckily, in binary that s just an AND gate! 0*0=0, 0*1=0, 1*0=0, 1*1=1 Generate a bunch of partial products

### CS173 Lecture B, November 3, 2015

CS173 Lecture B, November 3, 2015 Tandy Warnow November 3, 2015 CS 173, Lecture B November 3, 2015 Tandy Warnow Announcements Examlet 7 is a take-home exam, and is due November 10, 11:05 AM, in class.

Logic Gates are used to translate Boolean logic into circuits. In the abstract it is clear that we can build AND gates that perform the AND function and OR gates that perform the OR function and so on.

### STA Module 4 Probability Concepts. Rev.F08 1

STA 2023 Module 4 Probability Concepts Rev.F08 1 Learning Objectives Upon completing this module, you should be able to: 1. Compute probabilities for experiments having equally likely outcomes. 2. Interpret

### Binary addition example worked out

Binary addition example worked out Some terms are given here Exercise: what are these numbers equivalent to in decimal? The initial carry in is implicitly 0 1 1 1 0 (Carries) 1 0 1 1 (Augend) + 1 1 1 0

### Announcements Wednesday, September 20

Announcements Wednesday, September 20 WeBWorK 1.4, 1.5 are due on Wednesday at 11:59pm. The first midterm is on this Friday, September 22. Midterms happen during recitation. The exam covers through 1.5.

### God doesn t play dice. - Albert Einstein

ECE 450 Lecture 1 God doesn t play dice. - Albert Einstein As far as the laws of mathematics refer to reality, they are not certain; as far as they are certain, they do not refer to reality. Lecture Overview

### ASTR 4 Solar System Astronom y

Lectures: M thru F, :30-:0 am Location: De Anza Planetarium ASTR 4 Solar System Astronom y Marek Cichanski Office: S-a cichanskimarek@fhda.edu (40) 4-4 Office Hours: M thru F, :30 0:0 am In This Syllabus

### Discrete Probability and State Estimation

6.01, Fall Semester, 2007 Lecture 12 Notes 1 MASSACHVSETTS INSTITVTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science 6.01 Introduction to EECS I Fall Semester, 2007 Lecture 12 Notes

### Optimisation and Operations Research

Optimisation and Operations Research Lecture 15: The Greedy Heuristic Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/ School of

### Probabilistic Systems Analysis Spring 2018 Lecture 6. Random Variables: Probability Mass Function and Expectation

EE 178 Probabilistic Systems Analysis Spring 2018 Lecture 6 Random Variables: Probability Mass Function and Expectation Probability Mass Function When we introduce the basic probability model in Note 1,

### Intro to Information Theory

Intro to Information Theory Math Circle February 11, 2018 1. Random variables Let us review discrete random variables and some notation. A random variable X takes value a A with probability P (a) 0. Here

### Decision Trees. CSC411/2515: Machine Learning and Data Mining, Winter 2018 Luke Zettlemoyer, Carlos Guestrin, and Andrew Moore

Decision Trees Claude Monet, The Mulberry Tree Slides from Pedro Domingos, CSC411/2515: Machine Learning and Data Mining, Winter 2018 Luke Zettlemoyer, Carlos Guestrin, and Andrew Moore Michael Guerzhoy

### Probability and Inference. POLI 205 Doing Research in Politics. Populations and Samples. Probability. Fall 2015

Fall 2015 Population versus Sample Population: data for every possible relevant case Sample: a subset of cases that is drawn from an underlying population Inference Parameters and Statistics A parameter

### 9. Distance measures. 9.1 Classical information measures. Head Tail. How similar/close are two probability distributions? Trace distance.

9. Distance measures 9.1 Classical information measures How similar/close are two probability distributions? Trace distance Fidelity Example: Flipping two coins, one fair one biased Head Tail Trace distance

### Discrete Mathematics for CS Spring 2006 Vazirani Lecture 22

CS 70 Discrete Mathematics for CS Spring 2006 Vazirani Lecture 22 Random Variables and Expectation Question: The homeworks of 20 students are collected in, randomly shuffled and returned to the students.

### Announcements Monday, September 18

Announcements Monday, September 18 WeBWorK 1.4, 1.5 are due on Wednesday at 11:59pm. The first midterm is on this Friday, September 22. Midterms happen during recitation. The exam covers through 1.5. About

### PAST EXAM PAPER & MEMO N3 ABOUT THE QUESTION PAPERS:

EKURHULENI TECH COLLEGE. No. 3 Mogale Square, Krugersdorp. Website: www. ekurhulenitech.co.za Email: info@ekurhulenitech.co.za TEL: 011 040 7343 CELL: 073 770 3028/060 715 4529 PAST EXAM PAPER & MEMO N3

### 4.8 Huffman Codes. These lecture slides are supplied by Mathijs de Weerd

4.8 Huffman Codes These lecture slides are supplied by Mathijs de Weerd Data Compression Q. Given a text that uses 32 symbols (26 different letters, space, and some punctuation characters), how can we

### Lecture 1: September 25, A quick reminder about random variables and convexity

Information and Coding Theory Autumn 207 Lecturer: Madhur Tulsiani Lecture : September 25, 207 Administrivia This course will cover some basic concepts in information and coding theory, and their applications

### Bits. Chapter 1. Information can be learned through observation, experiment, or measurement.

Chapter 1 Bits Information is measured in bits, just as length is measured in meters and time is measured in seconds. Of course knowing the amount of information is not the same as knowing the information

### STAT/SOC/CSSS 221 Statistical Concepts and Methods for the Social Sciences. Random Variables

STAT/SOC/CSSS 221 Statistical Concepts and Methods for the Social Sciences Random Variables Christopher Adolph Department of Political Science and Center for Statistics and the Social Sciences University

### CPE100: Digital Logic Design I

Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu CPE100: Digital Logic Design I Midterm02 Review http://www.ee.unlv.edu/~b1morris/cpe100/ 2 Logistics Thursday Nov. 16 th In normal lecture (13:00-14:15)

### We set up the basic model of two-sided, one-to-one matching

Econ 805 Advanced Micro Theory I Dan Quint Fall 2009 Lecture 18 To recap Tuesday: We set up the basic model of two-sided, one-to-one matching Two finite populations, call them Men and Women, who want to

### Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14

CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten

### XOR - XNOR Gates. The graphic symbol and truth table of XOR gate is shown in the figure.

XOR - XNOR Gates Lesson Objectives: In addition to AND, OR, NOT, NAND and NOR gates, exclusive-or (XOR) and exclusive-nor (XNOR) gates are also used in the design of digital circuits. These have special

### U Logo Use Guidelines

COMP2610/6261 - Information Theory Lecture 22: Hamming Codes U Logo Use Guidelines Mark Reid and Aditya Menon logo is a contemporary n of our heritage. presents our name, d and our motto: arn the nature

### Using Microsoft Excel

Using Microsoft Excel Objective: Students will gain familiarity with using Excel to record data, display data properly, use built-in formulae to do calculations, and plot and fit data with linear functions.

### CSCI 2570 Introduction to Nanocomputing

CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication

### Physics 162a Quantum Mechanics

Physics 162a Quantum Mechanics 1 Introduction Syllabus for Fall 2009 This is essentially a standard first-year course in quantum mechanics, the basic language for describing physics at the atomic and subatomic

### Probability Experiments, Trials, Outcomes, Sample Spaces Example 1 Example 2

Probability Probability is the study of uncertain events or outcomes. Games of chance that involve rolling dice or dealing cards are one obvious area of application. However, probability models underlie

### Module 2. Basic Digital Building Blocks. Binary Arithmetic & Arithmetic Circuits Comparators, Decoders, Encoders, Multiplexors Flip-Flops

Module 2 asic Digital uilding locks Lecturer: Dr. Yongsheng Gao Office: Tech 3.25 Email: Web: Structure: Textbook: yongsheng.gao@griffith.edu.au maxwell.me.gu.edu.au 6 lecturers 1 tutorial 1 laboratory

### UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis

UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis Lecture 10 Class URL: http://vlsicad.ucsd.edu/courses/cse21-s14/ Lecture 10 Notes Midterm Good job overall! = 81; =

### MATH 3C: MIDTERM 1 REVIEW. 1. Counting

MATH 3C: MIDTERM REVIEW JOE HUGHES. Counting. Imagine that a sports betting pool is run in the following way: there are 20 teams, 2 weeks, and each week you pick a team to win. However, you can t pick

### ECE321 Electronics I

ECE321 Electronics I Lecture 1: Introduction to Digital Electronics Payman Zarkesh-Ha Office: ECE Bldg. 230B Office hours: Tuesday 2:00-3:00PM or by appointment E-mail: payman@ece.unm.edu Slide: 1 Textbook

### Math 381 Midterm Practice Problem Solutions

Math 381 Midterm Practice Problem Solutions Notes: -Many of the exercises below are adapted from Operations Research: Applications and Algorithms by Winston. -I have included a list of topics covered on

### Chapter 14. From Randomness to Probability. Copyright 2012, 2008, 2005 Pearson Education, Inc.

Chapter 14 From Randomness to Probability Copyright 2012, 2008, 2005 Pearson Education, Inc. Dealing with Random Phenomena A random phenomenon is a situation in which we know what outcomes could happen,