Information Theory. Master of Logic 2015/16 2nd Block Nov/Dec 2015
|
|
|
- Georgina Harmon
- 5 years ago
- Views:
Transcription
1 Information Theory Master of Logic 2015/16 2nd Block Nov/Dec 2015 Some of these slides are copied from or heavily inspired by the University of Illinois at Chicago, ECE 534: Elements of Information Theory course given in Fall 2013 by Natasha Devroye Thank you very much for the kind permission to re-use them here!
2 Christian Schaffner me pure mathematics at ETH Zurich PhD from Aarhus, Denmark research: quantum cryptography plays ultimate frisbee
3 Mathias Madsen your teaching assistant ex-phd working on cognitive science, broadly construed
4 Practicalities final grade consists of 50-50: average grade of the best 6 out of 7 weekly homework series final exam on Friday, Dec 18, 2015, 9:00-12:00 details on course homepage: inftheory/2015/
5 We expect from you be on time code of honor (do not cheat) focus ask questions! Expectations
6 Expectations We expect from you be on time code of honor (do not cheat) focus ask questions! You can expect from us be on time make clear what goals are listen to you and respond to requests keep website up to date
7 Expectations We expect from you be on time code of honor (do not cheat) focus ask questions! You can expect from us be on time make clear what goals are listen to you and respond to requests keep website up to date Why multitasking is bad for learning:
8 Questions?
9 What is communication?
10 What is communication? The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. - C.E. Shannon, 1948
11 What is communication? The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. - C.E. Shannon, 1948 Alice Bob
12 What is communication? The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. - C.E. Shannon, 1948 I want to send 1001 Alice Bob
13 What is communication? The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. - C.E. Shannon, 1948 I want to send 1001 I think Alice sent 1001 Alice Bob
14 History of (wireless) communication Smoke signals 1861: Maxwell s equations 1900: Marconi demonstrates wireless telegraph 1920s: Armstrong demonstrates FM radio mostly analog ad-hoc engineering, tailored to each application
15 History of (wireless) communication Smoke signals 1861: Maxwell s equations 1900: Marconi demonstrates wireless telegraph 1920s: Armstrong demonstrates FM radio mostly analog ad-hoc engineering, tailored to each application
16 History of (wireless) communication Smoke signals 1861: Maxwell s equations 1900: Guglielmo Marconi demonstrates wireless telegraph 1920s: Armstrong demonstrates FM radio mostly analog ad-hoc engineering, tailored to each application
17 History of (wireless) communication Smoke signals 1861: Maxwell s equations 1900: Marconi demonstrates wireless telegraph 1920s: Edwin Howard Armstrong demonstrates FM radio mostly analog ad-hoc engineering, tailored to each
18 Big Open Questions mostly analog ad-hoc engineering, tailored to each application is there a general methodology for designing communication systems? can we communicate reliably in noise? how fast can we communicate?
19 Claude Elwood Shannon Father of Information Theory Graduate of MIT 1940: An Algebra for Theoretical Genetics : Scientist at Bell Labs 1958: Professor at MIT: When he returned to MIT in 1958, he continued to threaten corridorwalkers on his unicycle, sometimes augmenting the hazard by juggling. No one was ever sure whether these activities were part of some new breakthrough or whether he just found them amusing. He worked, for example, on a motorized pogo-stick, which he claimed would mean he could abandon the unicycle so feared by his colleagues... juggling, unicycling, chess ultimate machine
20 History of (wireless) communication BITS! arguably, first to really define and use bits "He's one of the great men of the century. Without him, none of the things we know today would exist. The whole digital revolution started with him." -Neil Sloane, AT&T Fellow
21 Information Theory THE COIEF DIFFIOULTY ALOCE FOUOD OT FIRST WAS IN OAOAGING HER FLAOINGO: SHE SUCCEODEO ON GO OTIOG IOS BODY OUOKEO AOAO, COMFOROABLY EOOOGO, UNDER OER O OM, WITO OTS O O OS HANGIOG DOO O, BOT OENEOAO OY, OUST AS SO O HOD OOT OTS O OCK NOCEO O SOROIGHTEOEO O OT, ANO WOS O O ONG TO OIOE TO O HEDGEHOG O OLOW WOTH ITS O OAD, O O WOULO TWOST O OSEOF OOUO O ANO O O OK OP IN HOR OACO, O OTO OUO O A O O OZOED EO OREOSOOO O O O O SHO COUOD O O O O O O O O O OSO O OG O O O OAO OHO O O: AOD WHON O O O OAO OOO O O O O O O O DOO O, O OD O OS GOIOG O O BO O ON O O OIO, O O O OS O O OY O OOOOO O O O O O O O O O O O OT TO O OEOGO O O O O OD O OROLO O O O O O O OF, O O O O O O O O OHO O O O O O O O O O O O O O O O O O O
22 Introduced a new field: Information Theory What is communication? What is information? How much can we compress information? How fast can we communicate?
23 Main Contributions of Inf Theory Source coding source = random variable ultimate data compression limit is the source s entropy H
24 Main Contributions of Inf Theory Source coding source = random variable ultimate data compression limit is the source s entropy H Channel coding channel = conditional distributions ultimate transmission rate is the channel capacity C
25 Main Contributions of Inf Theory Source coding source = random variable ultimate data compression limit is the source s entropy H Channel coding channel = conditional distributions ultimate transmission rate is the channel capacity C Reliable communication possible H < C
26 Reactions to This Theory Engineers in disbelief Error free communication in noise eh? stuck in analogue world How to approach the predicted limits? Shannon says: can transmit at rates up to say 4Mbps over a certain channel without error. How to do it?
27 It Took 50 Years To Do It 50 s: algebraic codes 60 s 70 s: convolutional codes 80 s: iterative codes (LDPC, turbo codes) 2009: polar codes How to approach the predicted limits? review article by [Costello Forney 2006]
28 Applications of Information Theory Communication Theory Computer Science (e.g. in cryptography) Physics (thermodynamics) Philosophy of Science (Occam s Razor) Economics (investments) Biology (genetics, bio-informatics)
29 Topics Overview Entropy and Mutual Information Data Compression Coding Theory Entropy Diagrams Perfectly Secure Encryption Zero-Error Information Theory Channel-Coding Theorem Noisy-Channel Theorem
30 Questions?
31 Example: Letter Frequencies i a i p i 1 a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p q r s t u v w x y z Figure 2.1. Probability distribution over the 27 outcomes for a randomly selected letter in an English language document (estimated from The Frequently Asked Questions Manual for Linux ). The picture shows the probabilities by the areas of white squares. Book by David MacKay
32 Example: Letter Frequencies i a i p i 1 a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p q r s t u v w x y z Figure 2.1. Probability distribution over the 27 outcomes for a randomly selected letter in an English language document (estimated from The Frequently Asked Questions Manual for Linux ). The picture shows the probabilities by the areas of white squares. x a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p q r s t u v w x y z Figure 2.2. The probability distribution over the possible bigrams xy in an English language document, The Frequently Asked Questions Manual for Linux. Book by David MacKay y
33 Example: Surprisal Values from i a i p i h(p i ) 1 a b c d e f g h i j k l m n o p q r s t u v w x y z i p i log 2 1 p i 4.1 Table 2.9. Shannon information contents of the outcomes a z. Book by David MacKay
34 MacKay s Mnemonic convex concave
35 MacKay s Mnemonic convex concave
36 MacKay s Mnemonic convex concave
37 MacKay s Mnemonic convex convec-smile concave
38 MacKay s Mnemonic convex convec-smile concave conca-frown
39 Examples: Convex & Concave Functions x Book by David MacKay
40 Examples: Convex & Concave Functions x Book by David MacKay
41 Examples: Convex & Concave Functions x 2 e x Book by David MacKay
42 Examples: Convex & Concave Functions x 2 e x log 1 x Book by David MacKay
43 Examples: Convex & Concave Functions x 2 e x log 1 x x log x Book by David MacKay
44 Examples: Convex & Concave Functions x 2 e x log 1 x x log x Book by David MacKay
Examples of Noisy Channels
Examples of Noisy Channels analogue telephone line over which two modems communicate digital information a teacher mumbling at the board radio communication link between curiosity on Mars and earth reproducing
Example: Letter Frequencies
Example: Letter Frequencies i a i p i 1 a 0.0575 2 b 0.0128 3 c 0.0263 4 d 0.0285 5 e 0.0913 6 f 0.0173 7 g 0.0133 8 h 0.0313 9 i 0.0599 10 j 0.0006 11 k 0.0084 12 l 0.0335 13 m 0.0235 14 n 0.0596 15 o
Example: Letter Frequencies
Example: Letter Frequencies i a i p i 1 a 0.0575 2 b 0.0128 3 c 0.0263 4 d 0.0285 5 e 0.0913 6 f 0.0173 7 g 0.0133 8 h 0.0313 9 i 0.0599 10 j 0.0006 11 k 0.0084 12 l 0.0335 13 m 0.0235 14 n 0.0596 15 o
Example: Letter Frequencies
Example: Letter Frequencies i a i p i 1 a 0.0575 2 b 0.0128 3 c 0.0263 4 d 0.0285 5 e 0.0913 6 f 0.0173 7 g 0.0133 8 h 0.0313 9 i 0.0599 10 j 0.0006 11 k 0.0084 12 l 0.0335 13 m 0.0235 14 n 0.0596 15 o
Information in Biology
Information in Biology CRI - Centre de Recherches Interdisciplinaires, Paris May 2012 Information processing is an essential part of Life. Thinking about it in quantitative terms may is useful. 1 Living
Information in Biology
Lecture 3: Information in Biology Tsvi Tlusty, tsvi@unist.ac.kr Living information is carried by molecular channels Living systems I. Self-replicating information processors Environment II. III. Evolve
Outline of the Lecture. Background and Motivation. Basics of Information Theory: 1. Introduction. Markku Juntti. Course Overview
: Markku Juntti Overview The basic ideas and concepts of information theory are introduced. Some historical notes are made and the overview of the course is given. Source The material is mainly based on
Entropies & Information Theory
Entropies & Information Theory LECTURE I Nilanjana Datta University of Cambridge,U.K. See lecture notes on: http://www.qi.damtp.cam.ac.uk/node/223 Quantum Information Theory Born out of Classical Information
6.02 Fall 2011 Lecture #9
6.02 Fall 2011 Lecture #9 Claude E. Shannon Mutual information Channel capacity Transmission at rates up to channel capacity, and with asymptotically zero error 6.02 Fall 2011 Lecture 9, Slide #1 First
Quantum Teleportation Pt. 1
Quantum Teleportation Pt. 1 PHYS 500 - Southern Illinois University April 17, 2018 PHYS 500 - Southern Illinois University Quantum Teleportation Pt. 1 April 17, 2018 1 / 13 Types of Communication In the
3F1 Information Theory, Lecture 1
3F1 Information Theory, Lecture 1 Jossy Sayir Department of Engineering Michaelmas 2013, 22 November 2013 Organisation History Entropy Mutual Information 2 / 18 Course Organisation 4 lectures Course material:
Introduction to Information Theory
Introduction to Information Theory Impressive slide presentations Radu Trîmbiţaş UBB October 2012 Radu Trîmbiţaş (UBB) Introduction to Information Theory October 2012 1 / 19 Transmission of information
PERFECT SECRECY AND ADVERSARIAL INDISTINGUISHABILITY
PERFECT SECRECY AND ADVERSARIAL INDISTINGUISHABILITY BURTON ROSENBERG UNIVERSITY OF MIAMI Contents 1. Perfect Secrecy 1 1.1. A Perfectly Secret Cipher 2 1.2. Odds Ratio and Bias 3 1.3. Conditions for Perfect
Other Topics in Quantum Information
p. 1/23 Other Topics in Quantum Information In a course like this there is only a limited time, and only a limited number of topics can be covered. Some additional topics will be covered in the class projects.
Lecture 16 Oct 21, 2014
CS 395T: Sublinear Algorithms Fall 24 Prof. Eric Price Lecture 6 Oct 2, 24 Scribe: Chi-Kit Lam Overview In this lecture we will talk about information and compression, which the Huffman coding can achieve
Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information.
L65 Dept. of Linguistics, Indiana University Fall 205 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission rate
Dept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 28 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission
Chapter 2: Entropy and Mutual Information. University of Illinois at Chicago ECE 534, Natasha Devroye
Chapter 2: Entropy and Mutual Information Chapter 2 outline Definitions Entropy Joint entropy, conditional entropy Relative entropy, mutual information Chain rules Jensen s inequality Log-sum inequality
1 Ex. 1 Verify that the function H(p 1,..., p n ) = k p k log 2 p k satisfies all 8 axioms on H.
Problem sheet Ex. Verify that the function H(p,..., p n ) = k p k log p k satisfies all 8 axioms on H. Ex. (Not to be handed in). looking at the notes). List as many of the 8 axioms as you can, (without
A Simple Introduction to Information, Channel Capacity and Entropy
A Simple Introduction to Information, Channel Capacity and Entropy Ulrich Hoensch Rocky Mountain College Billings, MT 59102 hoenschu@rocky.edu Friday, April 21, 2017 Introduction A frequently occurring
Lecture 1: Shannon s Theorem
Lecture 1: Shannon s Theorem Lecturer: Travis Gagie January 13th, 2015 Welcome to Data Compression! I m Travis and I ll be your instructor this week. If you haven t registered yet, don t worry, we ll work
9. Distance measures. 9.1 Classical information measures. Head Tail. How similar/close are two probability distributions? Trace distance.
9. Distance measures 9.1 Classical information measures How similar/close are two probability distributions? Trace distance Fidelity Example: Flipping two coins, one fair one biased Head Tail Trace distance
Procedure for Setting Goals for an Introductory Physics Class
Procedure for Setting Goals for an Introductory Physics Class Pat Heller, Ken Heller, Vince Kuo University of Minnesota Important Contributions from Tom Foster, Francis Lawrenz Details at http://groups.physics.umn.edu/physed
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I
(Classical) Information Theory III: Noisy channel coding
(Classical) Information Theory III: Noisy channel coding Sibasish Ghosh The Institute of Mathematical Sciences CIT Campus, Taramani, Chennai 600 113, India. p. 1 Abstract What is the best possible way
AQI: Advanced Quantum Information Lecture 6 (Module 2): Distinguishing Quantum States January 28, 2013
AQI: Advanced Quantum Information Lecture 6 (Module 2): Distinguishing Quantum States January 28, 2013 Lecturer: Dr. Mark Tame Introduction With the emergence of new types of information, in this case
Computing and Communications 2. Information Theory -Entropy
1896 1920 1987 2006 Computing and Communications 2. Information Theory -Entropy Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2017, Autumn 1 Outline Entropy Joint entropy
Lecture 1: Introduction, Entropy and ML estimation
0-704: Information Processing and Learning Spring 202 Lecture : Introduction, Entropy and ML estimation Lecturer: Aarti Singh Scribes: Min Xu Disclaimer: These notes have not been subjected to the usual
Noisy channel communication
Information Theory http://www.inf.ed.ac.uk/teaching/courses/it/ Week 6 Communication channels and Information Some notes on the noisy channel setup: Iain Murray, 2012 School of Informatics, University
Introduction to Information Theory. Part 4
Introduction to Information Theory Part 4 A General Communication System CHANNEL Information Source Transmitter Channel Receiver Destination 10/2/2012 2 Information Channel Input X Channel Output Y 10/2/2012
An introduction to basic information theory. Hampus Wessman
An introduction to basic information theory Hampus Wessman Abstract We give a short and simple introduction to basic information theory, by stripping away all the non-essentials. Theoretical bounds on
A New Wireless Quantum Key Distribution Protocol based on Authentication And Bases Center (AABC)
A New Wireless Quantum Key Distribution Protocol based on Authentication And Bases Center (AABC) Majid Alshammari and Khaled Elleithy Department of Computer Science and Engineering University of Bridgeport
! Where are we on course map? ! What we did in lab last week. " How it relates to this week. ! Compression. " What is it, examples, classifications
Lecture #3 Compression! Where are we on course map?! What we did in lab last week " How it relates to this week! Compression " What is it, examples, classifications " Probability based compression # Huffman
Channel Coding: Zero-error case
Channel Coding: Zero-error case Information & Communication Sander Bet & Ismani Nieuweboer February 05 Preface We would like to thank Christian Schaffner for guiding us in the right direction with our
Lecture 1. Introduction
Lecture 1. Introduction What is the course about? Logistics Questionnaire Dr. Yao Xie, ECE587, Information Theory, Duke University What is information? Dr. Yao Xie, ECE587, Information Theory, Duke University
Seminar 1. Introduction to Quantum Computing
Seminar 1 Introduction to Quantum Computing Before going in I am also a beginner in this field If you are interested, you can search more using: Quantum Computing since Democritus (Scott Aaronson) Quantum
Simple Math: Cryptography
1 Introduction Simple Math: Cryptography This section develops some mathematics before getting to the application. The mathematics that I use involves simple facts from number theory. Number theory is
Lecture 18: Quantum Information Theory and Holevo s Bound
Quantum Computation (CMU 1-59BB, Fall 2015) Lecture 1: Quantum Information Theory and Holevo s Bound November 10, 2015 Lecturer: John Wright Scribe: Nicolas Resch 1 Question In today s lecture, we will
ECE INFORMATION THEORY. Fall 2011 Prof. Thinh Nguyen
1 ECE 566 - INFORMATION THEORY Fall 2011 Prof. Thinh Nguyen LOGISTICS Prerequisites: ECE 353, strong mathematical background Office hours: M, W 11-Noon, KEC3115 Course Content: Entropy, Relative Entropy,
Entanglement and information
Ph95a lecture notes for 0/29/0 Entanglement and information Lately we ve spent a lot of time examining properties of entangled states such as ab è 2 0 a b è Ý a 0 b è. We have learned that they exhibit
Lecture 11: Quantum Information III - Source Coding
CSCI5370 Quantum Computing November 25, 203 Lecture : Quantum Information III - Source Coding Lecturer: Shengyu Zhang Scribe: Hing Yin Tsang. Holevo s bound Suppose Alice has an information source X that
Historical Perspectives John von Neumann ( )
Historical Perspectives John von Neumann (1903-1957) Contributed to set theory, functional analysis, quantum mechanics, ergodic theory, economics, geometry, hydrodynamics, statistics, analysis, measure
Quantum Wireless Sensor Networks
Quantum Wireless Sensor Networks School of Computing Queen s University Canada ntional Computation Vienna, August 2008 Main Result Quantum cryptography can solve the problem of security in sensor networks.
Introduction to Information Theory. Part 2
Introduction to Information Theory Part 2 1 A General Communication System CHANNEL Information Source Transmitter Channel Receiver Destination 2 Information: Definition Information is quantified using
Analysis of the Influenceof the Rate of Spies Measure on the Quantum Transmission
Science Journal Of Mathematics and Statistics ISSN: 2276-6324 http://www.sjpub.org/sjms.html Author(s) 2012. CC Attribution 3.0 License. Research Article Published By Science Journal Publication International
CSCI 2570 Introduction to Nanocomputing
CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication
Quantum Cryptography
Quantum Cryptography Christian Schaffner Research Center for Quantum Software Institute for Logic, Language and Computation (ILLC) University of Amsterdam Centrum Wiskunde & Informatica Winter 17 QuantumDay@Portland
Error Reconciliation in QKD. Distribution
Error Reconciliation in Quantum Key Distribution Richard P. Brent MSI, ANU 1 October 2009 Abstract The problem of "error reconciliation" arises in Quantum Cryptography, which is more accurately described
Lecture 6: Quantum error correction and quantum capacity
Lecture 6: Quantum error correction and quantum capacity Mark M. Wilde The quantum capacity theorem is one of the most important theorems in quantum hannon theory. It is a fundamentally quantum theorem
Lecture 1: Perfect Secrecy and Statistical Authentication. 2 Introduction - Historical vs Modern Cryptography
CS 7880 Graduate Cryptography September 10, 2015 Lecture 1: Perfect Secrecy and Statistical Authentication Lecturer: Daniel Wichs Scribe: Matthew Dippel 1 Topic Covered Definition of perfect secrecy One-time
Quantum Information Types
qitd181 Quantum Information Types Robert B. Griffiths Version of 6 February 2012 References: R. B. Griffiths, Types of Quantum Information, Phys. Rev. A 76 (2007) 062320; arxiv:0707.3752 Contents 1 Introduction
Introduction to Information Theory. B. Škorić, Physical Aspects of Digital Security, Chapter 2
Introduction to Information Theory B. Škorić, Physical Aspects of Digital Security, Chapter 2 1 Information theory What is it? - formal way of counting information bits Why do we need it? - often used
An Introduction. Dr Nick Papanikolaou. Seminar on The Future of Cryptography The British Computer Society 17 September 2009
An Dr Nick Papanikolaou Research Fellow, e-security Group International Digital Laboratory University of Warwick http://go.warwick.ac.uk/nikos Seminar on The Future of Cryptography The British Computer
Quantum Entanglement and Cryptography. Deepthi Gopal, Caltech
+ Quantum Entanglement and Cryptography Deepthi Gopal, Caltech + Cryptography Concisely: to make information unreadable by anyone other than the intended recipient. The sender of a message scrambles/encrypts
Introduction to Information Theory. Part 3
Introduction to Information Theory Part 3 Assignment#1 Results List text(s) used, total # letters, computed entropy of text. Compare results. What is the computed average word length of 3 letter codes
Dan Boneh. Introduction. Course Overview
Online Cryptography Course Introduction Course Overview Welcome Course objectives: Learn how crypto primitives work Learn how to use them correctly and reason about security My recommendations: Take notes
10 - February, 2010 Jordan Myronuk
10 - February, 2010 Jordan Myronuk Classical Cryptography EPR Paradox] The need for QKD Quantum Bits and Entanglement No Cloning Theorem Polarization of Photons BB84 Protocol Probability of Qubit States
Quantum Error Correcting Codes and Quantum Cryptography. Peter Shor M.I.T. Cambridge, MA 02139
Quantum Error Correcting Codes and Quantum Cryptography Peter Shor M.I.T. Cambridge, MA 02139 1 We start out with two processes which are fundamentally quantum: superdense coding and teleportation. Superdense
Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 08 Shannon s Theory (Contd.)
Problem Set: TT Quantum Information
Problem Set: TT Quantum Information Basics of Information Theory 1. Alice can send four messages A, B, C, and D over a classical channel. She chooses A with probability 1/, B with probability 1/4 and C
Network Coding and Schubert Varieties over Finite Fields
Network Coding and Schubert Varieties over Finite Fields Anna-Lena Horlemann-Trautmann Algorithmics Laboratory, EPFL, Schweiz October 12th, 2016 University of Kentucky What is this talk about? 1 / 31 Overview
The Laws of Cryptography Zero-Knowledge Protocols
26 The Laws of Cryptography Zero-Knowledge Protocols 26.1 The Classes NP and NP-complete. 26.2 Zero-Knowledge Proofs. 26.3 Hamiltonian Cycles. An NP-complete problem known as the Hamiltonian Cycle Problem
Information Complexity and Applications. Mark Braverman Princeton University and IAS FoCM 17 July 17, 2017
Information Complexity and Applications Mark Braverman Princeton University and IAS FoCM 17 July 17, 2017 Coding vs complexity: a tale of two theories Coding Goal: data transmission Different channels
Shannon's Theory of Communication
Shannon's Theory of Communication An operational introduction 5 September 2014, Introduction to Information Systems Giovanni Sileno g.sileno@uva.nl Leibniz Center for Law University of Amsterdam Fundamental
Secrecy and the Quantum
Secrecy and the Quantum Benjamin Schumacher Department of Physics Kenyon College Bright Horizons 35 (July, 2018) Keeping secrets Communication Alice sound waves, photons, electrical signals, paper and
A Mathematical Theory of Communication
A Mathematical Theory of Communication Ben Eggers Abstract This paper defines information-theoretic entropy and proves some elementary results about it. Notably, we prove that given a few basic assumptions
U.C. Berkeley CS276: Cryptography Luca Trevisan February 5, Notes for Lecture 6
U.C. Berkeley CS276: Cryptography Handout N6 Luca Trevisan February 5, 2009 Notes for Lecture 6 Scribed by Ian Haken, posted February 8, 2009 Summary The encryption scheme we saw last time, based on pseudorandom
6.02 Fall 2012 Lecture #1
6.02 Fall 2012 Lecture #1 Digital vs. analog communication The birth of modern digital communication Information and entropy Codes, Huffman coding 6.02 Fall 2012 Lecture 1, Slide #1 6.02 Fall 2012 Lecture
CHE 717, Chemical Reaction Engineering, Fall 2016 COURSE SYLLABUS
Page 1 of 5 CHE 717, Chemical Reaction Engineering, Fall 2016 COURSE SYLLABUS engineeringonline.ncsu.edu/onlinecourses/coursehomepages/fall-2016/che717.html Instructor: Prof. Jason Haugh 3100 Partners
Physics is becoming too difficult for physicists. David Hilbert (mathematician)
Physics is becoming too difficult for physicists. David Hilbert (mathematician) Simple Harmonic Oscillator Credit: R. Nave (HyperPhysics) Particle 2 X 2-Particle wave functions 2 Particles, each moving
Performance-based Security for Encoding of Information Signals. FA ( ) Paul Cuff (Princeton University)
Performance-based Security for Encoding of Information Signals FA9550-15-1-0180 (2015-2018) Paul Cuff (Princeton University) Contributors Two students finished PhD Tiance Wang (Goldman Sachs) Eva Song
PERFECTLY secure key agreement has been studied recently
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 2, MARCH 1999 499 Unconditionally Secure Key Agreement the Intrinsic Conditional Information Ueli M. Maurer, Senior Member, IEEE, Stefan Wolf Abstract
HEAT AND THERMODYNAMICS PHY 522 Fall, 2010
HEAT AND THERMODYNAMICS PHY 522 Fall, 2010 I. INSTRUCTOR Professor Lance De Long Office: CP363 (257-4775) Labs: CP75, CP158 (257-8883), ASTeCC A041 Office Hours: M 10:30-11:30 a.m.; T 8:30-9:30 a.m. II.
Quantum sampling of mixed states
Quantum sampling of mixed states Philippe Lamontagne January 7th Philippe Lamontagne Quantum sampling of mixed states January 7th 1 / 9 The setup Philippe Lamontagne Quantum sampling of mixed states January
to mere bit flips) may affect the transmission.
5 VII. QUANTUM INFORMATION THEORY to mere bit flips) may affect the transmission. A. Introduction B. A few bits of classical information theory Information theory has developed over the past five or six
Lecture: Quantum Information
Lecture: Quantum Information Transcribed by: Crystal Noel and Da An (Chi Chi) November 10, 016 1 Final Proect Information Find an issue related to class you are interested in and either: read some papers
Introduction. Outline. EE3321 Electromagnetic Field Theory. Welcome! About this class Rules and syllabus Let s get started! 9/19/2018. E t.
D v B 0 B E t D H J t D B E H Introduction EE3321 Electromagnetic Field Theory Outline Welcome! About this class Rules and syllabus Let s get started! Slide 2 1 Welcome Course Information http://emlab.utep.edu/ee3321emf.htm
Information Theory and Coding Techniques: Chapter 1.1. What is Information Theory? Why you should take this course?
Information Theory and Coding Techniques: Chapter 1.1 What is Information Theory? Why you should take this course? 1 What is Information Theory? Information Theory answers two fundamental questions in
COS 341: Discrete Mathematics
COS 341: Discrete Mathematics Midterm Exam Fall 2006 Print your name General directions: This exam is due on Monday, November 13 at 4:30pm. Late exams will not be accepted. Exams must be submitted in hard
Midterm Exam Information Theory Fall Midterm Exam. Time: 09:10 12:10 11/23, 2016
Midterm Exam Time: 09:10 12:10 11/23, 2016 Name: Student ID: Policy: (Read before You Start to Work) The exam is closed book. However, you are allowed to bring TWO A4-size cheat sheet (single-sheet, two-sided).
Ma/CS 6a Class 1. Course Details
Ma/CS 6a Class 1 By Adam Sheffer Course Details Adam Sheffer. adamsh@caltech.edu 1:00 Monday, Wednesday, and Friday. http://www.math.caltech.edu/~2014-15/1term/ma006a/ 1 Course Structure No exam! Grade
Communication Theory II
Communication Theory II Lecture 11: Information Theory Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt March 15 th, 2015 1 Midterm Degrees Exam Type of Assignment Max. Final Oral* Oral
Chemistry SCI1100. Course Description. Rationale. Prerequisite. Measurable Learning Outcomes
Chemistry SCI1100 Course Description Chemistry is the investigation of atomic and molecular-level properties and interactions. The course begins with properties of matter, atomic structure, and basic atomic
STATISTICAL AND THERMAL PHYSICS
Phys 362 Spring 2015 STATISTICAL AND THERMAL PHYSICS Phys 362 Spring 2015 Instructor: Office: Professor David Collins WS 228B Phone: 248-1787 email: Office Hours: dacollin@coloradomesa.edu M 9 10 am, 11
Great Ideas in Theoretical Computer Science
15-251 Great Ideas in Theoretical Computer Science Lecture 1: Introduction to the course Instructors Aug 29th, 2017 Teaching Assistants What is computer science? What is theoretical computer science? What
Part I. Entropy. Information Theory and Networks. Section 1. Entropy: definitions. Lecture 5: Entropy
and Networks Lecture 5: Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/InformationTheory/ Part I School of Mathematical Sciences, University
Welcome to Physics 211! General Physics I
Welcome to Physics 211! General Physics I Physics 211 Fall 2015 Lecture 01-1 1 Physics 215 Honors & Majors Are you interested in becoming a physics major? Do you have a strong background in physics and
TECHNISCHE UNIVERSITEIT EINDHOVEN Faculty of Mathematics and Computer Science Exam Cryptology, Friday 25 January 2019
Faculty of Mathematics and Computer Science Exam Cryptology, Friday 25 January 2019 Name : TU/e student number : Exercise 1 2 3 4 5 total points Notes: Please hand in all sheets at the end of the exam.
Ma/CS 6b Class 24: Error Correcting Codes
Ma/CS 6b Class 24: Error Correcting Codes By Adam Sheffer Communicating Over a Noisy Channel Problem. We wish to transmit a message which is composed of 0 s and 1 s, but noise might accidentally flip some
Constructing Polar Codes Using Iterative Bit-Channel Upgrading. Arash Ghayoori. B.Sc., Isfahan University of Technology, 2011
Constructing Polar Codes Using Iterative Bit-Channel Upgrading by Arash Ghayoori B.Sc., Isfahan University of Technology, 011 A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree
CS120, Quantum Cryptography, Fall 2016
CS10, Quantum Cryptography, Fall 016 Homework # due: 10:9AM, October 18th, 016 Ground rules: Your homework should be submitted to the marked bins that will be by Annenberg 41. Please format your solutions
Bits. Chapter 1. Information can be learned through observation, experiment, or measurement.
Chapter 1 Bits Information is measured in bits, just as length is measured in meters and time is measured in seconds. Of course knowing the amount of information is not the same as knowing the information
AMIS 3600H Accounting Information Systems Spring 2016 Fisher 406
AMIS 3600H Accounting Information Systems John Fellingham Spring 2016 Fisher 406 Office hours: TBA General Description It is said we live in an information age, and the father of the information age is
PHYS Statistical Mechanics I Course Outline
PHYS 449 - Statistical Mechanics I Course Outline There is no official textbook for this course. Suggested References: An Introduction to Thermal Physics, by Daniel V. Schroeder: this was the textbook
D CUME N TAT ION C 0 R P 0 R A T E D 2,21 CONNECTICUT AVENUE, N. W,
44 D CUME N TAT ION C 0 R P 0 R A T E D 2,21 CONNECTICUT AVENUE, N. W, rs WlLISTS IN BASIC AND APPLIED INFORMATION THEORY WASHINGTON 8, D. C. C 0 L U M B I A 5-4 5 7 7 SOMMUNICATION THEORY AND STORAGE
Physics 18, Introductory Physics I for Biological Sciences Spring 2010
Physics 18 page 1/6 Physics 18, Introductory Physics I for Biological Sciences Spring 2010 - Course Description - Instructor: Dr. Derrick Kiley Office: AOB 176; Office Phone 209 228-3076 E-mail Address:
1 Background on Information Theory
Review of the book Information Theory: Coding Theorems for Discrete Memoryless Systems by Imre Csiszár and János Körner Second Edition Cambridge University Press, 2011 ISBN:978-0-521-19681-9 Review by
TECHNISCHE UNIVERSITEIT EINDHOVEN Faculty of Mathematics and Computer Science Exam Cryptology, Tuesday 30 October 2018
Faculty of Mathematics and Computer Science Exam Cryptology, Tuesday 30 October 2018 Name : TU/e student number : Exercise 1 2 3 4 5 total points Notes: Please hand in all sheets at the end of the exam.
CIS 6930/4930 Computer and Network Security. Topic 5.2 Public Key Cryptography
CIS 6930/4930 Computer and Network Security Topic 5.2 Public Key Cryptography 1 Diffie-Hellman Key Exchange 2 Diffie-Hellman Protocol For negotiating a shared secret key using only public communication
Quantum Information Theory and Cryptography
Quantum Information Theory and Cryptography John Smolin, IBM Research IPAM Information Theory A Mathematical Theory of Communication, C.E. Shannon, 1948 Lies at the intersection of Electrical Engineering,