Lecture 1: Basic problems of coding theory
|
|
- John Lawson
- 5 years ago
- Views:
Transcription
1 Lecture 1: Basic problems of codig theory Error-Correctig Codes (Sprig 016) Rutgers Uiversity Swastik Kopparty Scribes: Abhishek Bhrushudi & Aditya Potukuchi Admiistrivia was discussed at the begiig of this lecture. All the iformatio about the course ca be foud o the course web page: sk133/courses/codes- S16/. 1 Basic cocepts, defiitios, ad facts 1.1 Defiitios For most part of the course, will deote a parameter takig values i N. Defiitio 1 (Hammig distace). The Hammig distace betwee x, y {0, 1} is defied as the umber of coordiates i which x ad y differ. We shall deote the Hammig distace betwee x ad y by (x, y). A related otio is that of Hammig weight: Defiitio (Hammig weight). For x {0, 1}, the Hammig weight of x, deoted by wt(x) or x, is defied as (x, 0), where 0 deotes the all zero vector. Eve though we defied the above cocepts for strigs over {0, 1}, oe ca exted them to strigs over a arbitrary alphabet Σ i a atural maer. The reader ca easily check the followig: Fact 3. For x, y, z {0, 1}, (x, y) + (y, z) (x, z). Also, (x, y) 0, ad equality holds iff x = y. Aother way of sayig this is that (.,.) is a metric o {0, 1}. We are ow i a positio to defie what a code with miimum distace d is: Defiitio 4 (Code with miimum distace d). A code with miimum distace d i {0, 1} is a subset C {0, 1} with the property that mi x,y C,x y (x, y) = d. d is called the miimum distace of the code C. Note that the otio of miimum distace is well defied for ay subset of {0, 1}. Before we go o, we itroduce yet aother useful cocept. Defiitio 5 (Hammig ball). The Hammig ball of radius r cetered aroud c {0, 1} is the set {y {0, 1}, (y, c) r}. We will deote it by B (c, r), though it s typical to drop the i the subscript whe it s clear that we are workig over {0, 1}. 1
2 1. A short discussio o error correctio What does error correctio have to do with the combiatorial objects defied above? Let s cosider the followig sceario: Alice is o Earth ad Bob s o the Moo. They are sedig messages to each other, where the messages come from a subset C of {0, 1}. Give the huge distace betwee them, it s coceivable that wheever a message is set, some of its bits get corrupted (flipped). As we shall see later, it s reasoable to assume that wheever a message x C {0, 1} is trasmitted, at most t of its bits get corrupted. Alice ad Bob wat to be able to recover the origial message from a corrupted oe. Let us suppose that the set C had miimum distace d. How much error ca be tolerated, i.e. how large ca t be? Claim 6. t d 1 Proof. Suppose d < t + 1. Let c 1 ad c be i C such that (c 1, c ) = d. It follows that there is a strig x {0, 1} such that (x, c 1 ) t ad (x, c ) t. Now suppose Alice seds c 1 to Bob ad it gets corrupted with t errors ad becomes x whe it reaches Bob. Bob has o way to tell whether Alice had set c 1 or c sice both ca be corrupted with t errors ad be made ito x. But what happes whe t d 1? Ca the origial codeword be recovered? The followig fact, which follows trivially from the triagle iequality, tells us that recovery is possible i this regime: Fact 7. Let C be a code with miimum distace d. For ay c, c C, c c, we have that B(c, d 1 ) is disjoit from B(c, d 1 ). Proof. We will give a proof by cotradictio. Suppose x B(c, d 1 ) B(c, d 1 ). The (x, c) d 1 ad (x, c ) d 1, ad by triagle iequality, we have (c, c ) d 1 < d, which is a cotradictio. Thus, we wat the miimum distace to be as large as possible i order to be able to tolerate a large umber of errors. But the oe might argue, why ot just take C to be the set cotaiig the all zero strig ad the all oe strig, makig the distace? The reaso is that we ot oly wat large distace, but also wat C to be large. Why? Let s say that Alice ad Bob have come up with a way to idetify the itegers 1,..., C with the elemets of C. Now wheever Alice has a umber i her mid that she wats to covey to Bob, all she has to do is to sed the elemet of C correspodig to that umber. But, for doig this, she has to sed a strig of legth. It is easy to see that itegers betwee 1 ad C ca be represeted usig roughly log C bits, ad so if C <<, the Alice ad Bob are beig wasteful - the effective amout of iformatio beig set i the bits is log C <<. 1.3 The mai questios The above discussio raises the followig questios: 1. Give, d, how large ca C {0, 1} be such that C has miimum distace d?
3 . Give, d, how ca oe efficietly costruct a code i {0, 1} with miimum distace d? 3. Give, d, a code C {0, 1} with miimum distace d, ad a x {0, 1} with the property that there is a (uique) c C such that x B(c, d 1 ), how ca oe efficietly decode x, i.e. how does oe efficietly fid c? For this lecture, we will mostly focus our attetio o the first questio. Basic results We begi by givig some basic existece ad impossibility results about the umber of codewords i a code with miimum distace d. We will always be iterested i the asymptotics of. Particularly iterestig choices of d are (1) costat distace d = O(1), ad () costat relative distace d = δ..1 The volume boud Set r = d 1. It follows from Fact 7 that c C B(c, r) = c C B(c, r) = C B(r). Also ad so we have which gives us B(c, r) {0, 1} c C B(c, r) c C C B(r). Thus, C (d 1)/ i=0 This boud is kow as the volume boud or the Hammig boud. ( i). The greedy costructio A greedy approach: Cosider the followig greedy process: Set S = {0, 1} ad let C =. As log as S, pick a poit v S ad add v to C. S = S \ B(v, d 1). Remove B(v, d 1) from S, i.e. 3
4 Notice that we are removig at most d 1 ( ) i=0 i poits from S i every iteratio. Sice we add oe elemet to C i each iteratio, we have that Usig the fact that d = O(1), we get C d 1 i=0 C = Ω( d 1 ) Note that this boud does ot match the volume boud that we proved. It s coceivable that a modificatio of this approach might give better bouds though this has eluded everyoe till date. ( i).3 A probabilistic costructio It turs out that radomly chose codes also achieve roughly the same parameters as the greedy costructio. However, some care is eeded i the radom choice, as show by the followig failed attempt. See the homework ad the ext lecture for more o this. Naive probabilistic approach: A aive probabilistic approach is to pick K vectors, x 1,..., x K, i {0, 1} uiformly ad idepedetly, ad set C = {x 1,..., x K }. Fact 8 (Birthday Paradox). If we pick Θ( N) umbers betwee 1 ad N uiformly ad idepedetly the, with probability 0.9, at least two of them will be the same. Thus, if K = Θ( / ), with probability 0.99, it would be the case that x i = x j for some i j. I fact, it s ot hard to see that, eve if we do t worry about duplicatios, with high probability, two of the strigs will ed up beig i the same ball of radius d 1, which would thwart our attempt. A probabilistic approach with alteratios: This approach ivolves pickig strigs uiformly ad idepedetly as above, followed by doig some modificatios to the set i order to get the distace property. [See the first homework set] 3 Costat distace codes Cosider the case whe is a growig parameter, ad d is a fixed costat, i.e. d = O(1). I this case, the volume packig boud says that: ( ) C O (d 1)/. The greedy code costructio produces a code C with ( C Ω These bouds are roughly i the same ball-park, but they are still sigificatly off. I a later lecture we will see that i fact the volume packig boud is tight: this is by a remarkable algebraic family of codes kow as BCH codes. d 1 ). 4
5 4 The d = 3 case: the Hammig code For d = 3, the volume boud becomes ( ) C = O We will show a costructio that gives a code with C = Θ( /), thus resolvig our questio for d = 3. This code is kow as the Hammig code, ad is due to Richard Hammig who also showed the volume boud. We idetify {0, 1} with the field F, ad thik of the code as a subset of F. Let l N be a parameter, ad H be the l ( l 1) matrix whose colums cosist of all o-zero vectors i F l. We set = l 1 ad defie our code as C = {c F ; Hc = 0} The reader ca check that C is a subspace of F whose dimesio is l1. Thus, we have ( ) C = l = + 1 = Θ All that remais to be show is that the miimum distace of C is 3. We will do this by showig that the miimum distace of C caot be 1 or. Before we show this, we will take a slight detour to talk about some properties of codes that are subspace i F. The followig is a easy fact: Fact 9. Let u, v, w F. The (u, v) = (u w, v w). If a code is a subspace of F, it is called a liear code. We will study more properties of liear codes i the ext lecture, but here is a fact that we will eed for ow: Fact 10. If C is a liear code with miimum distace d the mi c C wt(c) = d, where wt(.) deotes the Hammig weight fuctio. Proof. We will first show that mi c C wt(c) d. Sice C has miimum distace d there are codewords c 1, c C such that (c 1, c ) = d. Usig Fact 9, (c 1 c, 0) = d, which is the same as wt(c 1 c ) = d. Sice C is a liear code c 1 c C ad we have mi c C wt(c) d. We will ow show the other directio. Suppose there is a c C such that wt(c) < d. Sice C is a liear code ad hece a subspace, we have that 0 C. The, (c, 0) < d, which would be a cotradictio sice the miimum distace of C is d. We ow get back to showig that the miimum distace of the Hammig code is 3. By Fact 10, this is the same as showig that there are o codewords of Hammig weight less tha 3. Case i - there is a codeword c such that wt(c) = 1: From the defiitio of the Hammig code, we have Hc = 0. But this is ot possible sice Hc is a colum of H, ad all the colums of H are o-zero. 1 This follows from the rak-ullity theorem over F. The reader is urged to try ad prove the theorem over F if he or she has t doe so before. 5
6 Case ii - there is a codeword c such that wt(c) = : Agai, Hc = 0, ad Hc = v w, where v, w are distict colums of H. This meas v w = 0 or v = w, which is a cotradictio. This shows that the miimum distace of the Hammig code is at least 3. Codes that match the volume boud are called perfect codes ad the Hammig code is such a code for the d = 3 case. We ow try ad aswer the third questio i Sectio 1.3 for the Hammig code, i.e. give x {0, 1} such that (x, c) 1 for some c C, how do we efficietly fid c? Typically, efficietly here meas that the ruig time of our recovery algorithm should be polyomial i, the iput size. I the case of the Hammig code, polyomial time is trivial, sice oe ca try all 1-bit modificatios of x ad check if that resultig strig is i the code. This gives a quadratic time decodig algorithm. Below we give a liear time decodig algorithm. 1. Give x, compute Hx. This ivolves multiplyig a Θ(log ) matrix with a 1 vector, which ca be doe i time. Θ( log ).. If Hx = 0, the x C. Otherwise x = c + e i for some c C, where e i is the vector i F which is 0 everywhere except i the i th coordiate. The H(c + e i ) = Hc + He i = He i. 3. Note that He i is the i th colum of H. We ow wat to compute i give He i. This ca be doe by comparig He i with all the colums of H, ad takes time O( log ). 4. Oce i is determied, we ca compute c as x + e i. Thus, we have a algorithm to recover c that takes time Θ( log ). This ca be made Θ() by divide ad coquer: exercise! 5 More o liear Codes Recall that a code C F is said to be liear if it is a subspace of F. The defiitio of liear codes allows us to succictly represet the code by selectig a basis v 1, v,..., v k of C where k. It also allows a bijectio betwee {0, 1} k ad C as follows: for a σ {0, 1} k, the correspodig codeword is c(σ) = k i=1 σ 1v i F. To this extet, let G be a matrix with k rows ad colums, where the i th row is vi T. v T 1 v T G =... vk T The matrix G is called the Geerator Matrix for C. The map E : F k F give by E(m) = mg is called the Ecodig Map. Here, oe ormally thiks of m as a message (give by a biary strig) ad E(m) as the ecodig of m. F k Ecodig F Corrupt F Decodig F k 6
7 Sice liear codes are othig but vector spaces, we defie a Dual Code C of a liear code C F, aalogous to dual spaces, as follows: C := {y F x i y i = 0 for all x C} i=1 Observatio 11. (C ) = C Observatio 1. From the rak-ullity theorem, we have dim(c) + dim(c ) =. A Parity Check Matrix for a liear code C {0, 1} is a ( k) matrix with rows w T 1, wt,..., wt k, where w 1, w,..., w k are the basis vectors for C Observatio 13. c C iff Hc = 0 Observatio 14. GH T = 0 w T 1 w T H =... w k T The above observatio is the mai property of the Parity Check Matrix. Propositio 15. The followig are equivalet: 1. C has miimum distace at least d. Every ozero elemet of C has at least d o zero etries 3. Every d 1 colums of H are liearly idepedet. Proof. (1 ): Sice 0 C, we have that for every x C, where x 0, (x, 0) d. Therefore, x has at least d ozero etries. ( 3 ): Sice Hx = 0 implies x C, if the sum of some t < d colums i H was equal to 0, the we have a codeword that has t < d ozero etries, which is a cotradictio. (3 1 ): Suppose there existed x, y C such that (x, y) < d, we have Hx = Hy = 0, which gives us H(x + y) = 0. But sice x + y had less tha d ozero etries, the sum of < d colums i H equal to 0, which is a cotradictio 5.1 Larger Alphabets So far, we have thought of all of our messages, ad their ecodigs as biary strigs. However, i geeral, we eed ot be restricted by this coditio. Let Σ be ay fiite set of alphabet which 7
8 make up the cotet of the messages. We ca exted the defiitio of Hammig distace betwee two strigs x ad y i this alphabet to be the umber of poits where they differ. We ca also get a similar volume packig boud as before. Let C be code over the alphabet Σ that ca correct at most r errors. Let B (r) be the set of strigs i Σ, that differ i at most r poits from a give Σ strig. The, we have: C Σ B (r) Σ We ca also pick words greedily to get a lower boud o the umber of code words with miimum distace d: C Σ B (d) Σ However, to talk about liear codes, we ca oly use certai sizes of alphabet, i particular, if Σ is either a prime, or a prime power, we ca idetify Σ with a field F p or F p l, ad talk about liear codes as subspaces of F p or F p l. We ca defie a Geerator Matrix similar to the biary case as a k matrix G, where the rows are a basis of C. We also defie a dual code C as before, ad a Parity Check Matrix as a ( k) matrix whose rows are a basis of C. A few remarks: 1 Eve if you oly care about biary codes, the study codes over larger alphabet sizes is very importat, sice some of the best kow costructios of biary codes are by usig codes over larger alphabet sizes. Imposig the coditio of liearity o codes does ot seem to come with ay disadvatages. We do ot kow ay cosiderable differeces betwee codes ad liear codes. 6 Erasure Correctio I the regime of Erasure Correctio, a codeword is give by a strig x {0, 1}. However, istead of beig flipped, some bits are replaced with the symbol?. These ca be thought of as erasures, i.e., the bits i those locatio have bee erased. Our goal is to desig a code that is resiliet to at most t erasures. I other words, if C is a t erasure code, the for every c C, ad for every set of t coordiates, if we erase these t coordiates, the c should ot match with ay other strig i C i the remaiig t coordiates. Observatio 16. A code C is t erasure iff it has miimum distace at least t + 1. Observatio 17. Hammig code is erasure. 8
9 7 Costat relative distace Now we study the regime where the distace of the code is δ. The rate of a biary code C, give by log C, is a measure of how much iformatio each coordiate cotais (there are log C bits of iformatio eeded to specify a codeword i C, ad this iformatio is the writte i the form of bits). For a code over a geeral alphabet Σ, the rate is give by log Σ C. The Relative Miimum Distace (or simply relative distace) of a code C {0, 1} with miimum distace d is give by d. Before we study the relatio betwee the rate ad miimum distace of a code, we establish the followig useful estimate for the volume of the Hammig ball. For ay positive costat ρ < 1, we have: B (ρ) = Θ ( ) H(ρ) 1 where H(ρ) is called the Shao Etropy give by H(ρ) = ρ log ρ +(1 ρ) log 1 1 ρ. This itimate relatio to the volumes of Hammig balls is the reaso that the fuctio H shows up so much i the study of codes. The followig plot shows the relatio betwee ρ ad H(ρ). It is symmetric about ρ = 1/, where it takes its maximum value, H(1/) = 1, ad is 0 at ρ = 0, ad ρ = H(ρ) ρ 9
10 7.1 Upper Boud From the Volume Packig Boud, we have C = O H(δ/) Takig logarithms o both sides ad dividig by, we have ( ) δ R 1 H + o(1) 7. Lower Boud Usig the greedy approach as before, we get C B (δ 1) = Ω H(δ) Takig logarithms o both sides ad divig by, R 1 H(δ) + o(1) 7.3 The story so far.. So far, we kow that for a positive costat δ < 1, there caot exist codes of rate more tha 1 H(δ/) +o(1) (Volume Packig Boud), ad that there exist codes with rate at least 1 H(δ) +o(1). These are illustrated i the followig plot R r b r b δ 10
11 The upper boud is ot tight for every δ. For istace, we kow that there do ot exist codes with ozero rate with relative distace 3/4 (Exercise!). I the ext lecture, we will see that i fact, there are o codes with ozero rate with relative distace 1/, ad give a somewhat better overall upper boud. 11
The multiplicative structure of finite field and a construction of LRC
IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information1 Hash tables. 1.1 Implementation
Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationLecture 9: Pseudo-random generators against space bounded computation,
Lecture 9: Pseudo-radom geerators agaist space bouded computatio, Primality Testig Topics i Pseudoradomess ad Complexity (Sprig 2018) Rutgers Uiversity Swastik Kopparty Scribes: Harsha Tirumala, Jiyu Zhag
More information6.895 Essential Coding Theory October 20, Lecture 11. This lecture is focused in comparisons of the following properties/parameters of a code:
6.895 Essetial Codig Theory October 0, 004 Lecture 11 Lecturer: Madhu Suda Scribe: Aastasios Sidiropoulos 1 Overview This lecture is focused i comparisos of the followig properties/parameters of a code:
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationLecture 14: Graph Entropy
15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More information4 The Sperner property.
4 The Sperer property. I this sectio we cosider a surprisig applicatio of certai adjacecy matrices to some problems i extremal set theory. A importat role will also be played by fiite groups. I geeral,
More informationThe Growth of Functions. Theoretical Supplement
The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that
More informationUC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 17 Lecturer: David Wagner April 3, Notes 17 for CS 170
UC Berkeley CS 170: Efficiet Algorithms ad Itractable Problems Hadout 17 Lecturer: David Wager April 3, 2003 Notes 17 for CS 170 1 The Lempel-Ziv algorithm There is a sese i which the Huffma codig was
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationCHAPTER I: Vector Spaces
CHAPTER I: Vector Spaces Sectio 1: Itroductio ad Examples This first chapter is largely a review of topics you probably saw i your liear algebra course. So why cover it? (1) Not everyoe remembers everythig
More informationShannon s noiseless coding theorem
18.310 lecture otes May 4, 2015 Shao s oiseless codig theorem Lecturer: Michel Goemas I these otes we discuss Shao s oiseless codig theorem, which is oe of the foudig results of the field of iformatio
More informationLecture 9: Expanders Part 2, Extractors
Lecture 9: Expaders Part, Extractors Topics i Complexity Theory ad Pseudoradomess Sprig 013 Rutgers Uiversity Swastik Kopparty Scribes: Jaso Perry, Joh Kim I this lecture, we will discuss further the pseudoradomess
More informationsubcaptionfont+=small,labelformat=parens,labelsep=space,skip=6pt,list=0,hypcap=0 subcaption ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, 2/16/2016
subcaptiofot+=small,labelformat=pares,labelsep=space,skip=6pt,list=0,hypcap=0 subcaptio ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, /6/06. Self-cojugate Partitios Recall that, give a partitio λ, we may
More informationA Note on Matrix Rigidity
A Note o Matrix Rigidity Joel Friedma Departmet of Computer Sciece Priceto Uiversity Priceto, NJ 08544 Jue 25, 1990 Revised October 25, 1991 Abstract I this paper we give a explicit costructio of matrices
More informationLecture 11: Pseudorandom functions
COM S 6830 Cryptography Oct 1, 2009 Istructor: Rafael Pass 1 Recap Lecture 11: Pseudoradom fuctios Scribe: Stefao Ermo Defiitio 1 (Ge, Ec, Dec) is a sigle message secure ecryptio scheme if for all uppt
More information4.3 Growth Rates of Solutions to Recurrences
4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.
More informationLecture 9: Hierarchy Theorems
IAS/PCMI Summer Sessio 2000 Clay Mathematics Udergraduate Program Basic Course o Computatioal Complexity Lecture 9: Hierarchy Theorems David Mix Barrigto ad Alexis Maciel July 27, 2000 Most of this lecture
More informationCS284A: Representations and Algorithms in Molecular Biology
CS284A: Represetatios ad Algorithms i Molecular Biology Scribe Notes o Lectures 3 & 4: Motif Discovery via Eumeratio & Motif Represetatio Usig Positio Weight Matrix Joshua Gervi Based o presetatios by
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationMATH 205 HOMEWORK #2 OFFICIAL SOLUTION. (f + g)(x) = f(x) + g(x) = f( x) g( x) = (f + g)( x)
MATH 205 HOMEWORK #2 OFFICIAL SOLUTION Problem 2: Do problems 7-9 o page 40 of Hoffma & Kuze. (7) We will prove this by cotradictio. Suppose that W 1 is ot cotaied i W 2 ad W 2 is ot cotaied i W 1. The
More informationCS 330 Discussion - Probability
CS 330 Discussio - Probability March 24 2017 1 Fudametals of Probability 11 Radom Variables ad Evets A radom variable X is oe whose value is o-determiistic For example, suppose we flip a coi ad set X =
More informationBinary codes from graphs on triples and permutation decoding
Biary codes from graphs o triples ad permutatio decodig J. D. Key Departmet of Mathematical Scieces Clemso Uiversity Clemso SC 29634 U.S.A. J. Moori ad B. G. Rodrigues School of Mathematics Statistics
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationBertrand s Postulate
Bertrad s Postulate Lola Thompso Ross Program July 3, 2009 Lola Thompso (Ross Program Bertrad s Postulate July 3, 2009 1 / 33 Bertrad s Postulate I ve said it oce ad I ll say it agai: There s always a
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationCSE 202 Homework 1 Matthias Springer, A Yes, there does always exist a perfect matching without a strong instability.
CSE 0 Homework 1 Matthias Spriger, A9950078 1 Problem 1 Notatio a b meas that a is matched to b. a < b c meas that b likes c more tha a. Equality idicates a tie. Strog istability Yes, there does always
More information(3) If you replace row i of A by its sum with a multiple of another row, then the determinant is unchanged! Expand across the i th row:
Math 50-004 Tue Feb 4 Cotiue with sectio 36 Determiats The effective way to compute determiats for larger-sized matrices without lots of zeroes is to ot use the defiitio, but rather to use the followig
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationRecitation 4: Lagrange Multipliers and Integration
Math 1c TA: Padraic Bartlett Recitatio 4: Lagrage Multipliers ad Itegratio Week 4 Caltech 211 1 Radom Questio Hey! So, this radom questio is pretty tightly tied to today s lecture ad the cocept of cotet
More informationLecture 2: April 3, 2013
TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 2: April 3, 203 Scribe: Shubhedu Trivedi Coi tosses cotiued We retur to the coi tossig example from the last lecture agai: Example. Give,
More informationSome remarks for codes and lattices over imaginary quadratic
Some remarks for codes ad lattices over imagiary quadratic fields Toy Shaska Oaklad Uiversity, Rochester, MI, USA. Caleb Shor Wester New Eglad Uiversity, Sprigfield, MA, USA. shaska@oaklad.edu Abstract
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationTEACHER CERTIFICATION STUDY GUIDE
COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationNotes for Lecture 11
U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More informationECEN 655: Advanced Channel Coding Spring Lecture 7 02/04/14. Belief propagation is exact on tree-structured factor graphs.
ECEN 655: Advaced Chael Codig Sprig 014 Prof. Hery Pfister Lecture 7 0/04/14 Scribe: Megke Lia 1 4-Cycles i Gallager s Esemble What we already kow: Belief propagatio is exact o tree-structured factor graphs.
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 11
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract We will itroduce the otio of reproducig kerels ad associated Reproducig Kerel Hilbert Spaces (RKHS). We will cosider couple
More informationTHE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS
THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS DEMETRES CHRISTOFIDES Abstract. Cosider a ivertible matrix over some field. The Gauss-Jorda elimiatio reduces this matrix to the idetity
More informationLecture 4: Unique-SAT, Parity-SAT, and Approximate Counting
Advaced Complexity Theory Sprig 206 Lecture 4: Uique-SAT, Parity-SAT, ad Approximate Coutig Prof. Daa Moshkovitz Scribe: Aoymous Studet Scribe Date: Fall 202 Overview I this lecture we begi talkig about
More informationZeros of Polynomials
Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More informationHashing and Amortization
Lecture Hashig ad Amortizatio Supplemetal readig i CLRS: Chapter ; Chapter 7 itro; Sectio 7.. Arrays ad Hashig Arrays are very useful. The items i a array are statically addressed, so that isertig, deletig,
More informationM A T H F A L L CORRECTION. Algebra I 1 4 / 1 0 / U N I V E R S I T Y O F T O R O N T O
M A T H 2 4 0 F A L L 2 0 1 4 HOMEWORK ASSIGNMENT #4 CORRECTION Algebra I 1 4 / 1 0 / 2 0 1 4 U N I V E R S I T Y O F T O R O N T O P r o f e s s o r : D r o r B a r - N a t a Correctio Homework Assigmet
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationLecture 10: Universal coding and prediction
0-704: Iformatio Processig ad Learig Sprig 0 Lecture 0: Uiversal codig ad predictio Lecturer: Aarti Sigh Scribes: Georg M. Goerg Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved
More informationInjections, Surjections, and the Pigeonhole Principle
Ijectios, Surjectios, ad the Pigeohole Priciple 1 (10 poits Here we will come up with a sloppy boud o the umber of parethesisestigs (a (5 poits Describe a ijectio from the set of possible ways to est pairs
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More information# fixed points of g. Tree to string. Repeatedly select the leaf with the smallest label, write down the label of its neighbour and remove the leaf.
Combiatorics Graph Theory Coutig labelled ad ulabelled graphs There are 2 ( 2) labelled graphs of order. The ulabelled graphs of order correspod to orbits of the actio of S o the set of labelled graphs.
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationCardinality Homework Solutions
Cardiality Homework Solutios April 16, 014 Problem 1. I the followig problems, fid a bijectio from A to B (you eed ot prove that the fuctio you list is a bijectio): (a) A = ( 3, 3), B = (7, 1). (b) A =
More informationIntermediate Math Circles November 4, 2009 Counting II
Uiversity of Waterloo Faculty of Mathematics Cetre for Educatio i Mathematics ad Computig Itermediate Math Circles November 4, 009 Coutig II Last time, after lookig at the product rule ad sum rule, we
More informationLast time, we talked about how Equation (1) can simulate Equation (2). We asserted that Equation (2) can also simulate Equation (1).
6896 Quatum Complexity Theory Sept 23, 2008 Lecturer: Scott Aaroso Lecture 6 Last Time: Quatum Error-Correctio Quatum Query Model Deutsch-Jozsa Algorithm (Computes x y i oe query) Today: Berstei-Vazirii
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationREAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS
REAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS 18th Feb, 016 Defiitio (Lipschitz fuctio). A fuctio f : R R is said to be Lipschitz if there exists a positive real umber c such that for ay x, y i the domai
More informationSeptember 2012 C1 Note. C1 Notes (Edexcel) Copyright - For AS, A2 notes and IGCSE / GCSE worksheets 1
September 0 s (Edecel) Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationComputability and computational complexity
Computability ad computatioal complexity Lecture 4: Uiversal Turig machies. Udecidability Io Petre Computer Sciece, Åbo Akademi Uiversity Fall 2015 http://users.abo.fi/ipetre/computability/ 21. toukokuu
More informationHOMEWORK 2 SOLUTIONS
HOMEWORK SOLUTIONS CSE 55 RANDOMIZED AND APPROXIMATION ALGORITHMS 1. Questio 1. a) The larger the value of k is, the smaller the expected umber of days util we get all the coupos we eed. I fact if = k
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationThe Borel hierarchy classifies subsets of the reals by their topological complexity. Another approach is to classify them by size.
Lecture 7: Measure ad Category The Borel hierarchy classifies subsets of the reals by their topological complexity. Aother approach is to classify them by size. Filters ad Ideals The most commo measure
More informationLecture 7: October 18, 2017
Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationChapter IV Integration Theory
Chapter IV Itegratio Theory Lectures 32-33 1. Costructio of the itegral I this sectio we costruct the abstract itegral. As a matter of termiology, we defie a measure space as beig a triple (, A, µ), where
More informationSection 1.1. Calculus: Areas And Tangents. Difference Equations to Differential Equations
Differece Equatios to Differetial Equatios Sectio. Calculus: Areas Ad Tagets The study of calculus begis with questios about chage. What happes to the velocity of a swigig pedulum as its positio chages?
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationThe Random Walk For Dummies
The Radom Walk For Dummies Richard A Mote Abstract We look at the priciples goverig the oe-dimesioal discrete radom walk First we review five basic cocepts of probability theory The we cosider the Beroulli
More informationAnalysis of Algorithms. Introduction. Contents
Itroductio The focus of this module is mathematical aspects of algorithms. Our mai focus is aalysis of algorithms, which meas evaluatig efficiecy of algorithms by aalytical ad mathematical methods. We
More informationSection 11.8: Power Series
Sectio 11.8: Power Series 1. Power Series I this sectio, we cosider geeralizig the cocept of a series. Recall that a series is a ifiite sum of umbers a. We ca talk about whether or ot it coverges ad i
More informationThe Maximum-Likelihood Decoding Performance of Error-Correcting Codes
The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,
More informationChapter Vectors
Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it
More information3.2 Properties of Division 3.3 Zeros of Polynomials 3.4 Complex and Rational Zeros of Polynomials
Math 60 www.timetodare.com 3. Properties of Divisio 3.3 Zeros of Polyomials 3.4 Complex ad Ratioal Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered
More informationThe Binomial Theorem
The Biomial Theorem Robert Marti Itroductio The Biomial Theorem is used to expad biomials, that is, brackets cosistig of two distict terms The formula for the Biomial Theorem is as follows: (a + b ( k
More informationLecture 2: Uncomputability and the Haling Problem
CSE 200 Computability ad Complexity Wedesday, April 3, 2013 Lecture 2: Ucomputability ad the Halig Problem Istructor: Professor Shachar Lovett Scribe: Dogcai She 1 The Uiversal Turig Machie I the last
More information1. By using truth tables prove that, for all statements P and Q, the statement
Author: Satiago Salazar Problems I: Mathematical Statemets ad Proofs. By usig truth tables prove that, for all statemets P ad Q, the statemet P Q ad its cotrapositive ot Q (ot P) are equivalet. I example.2.3
More informationOPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES
OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES Peter M. Maurer Why Hashig is θ(). As i biary search, hashig assumes that keys are stored i a array which is idexed by a iteger. However, hashig attempts to bypass
More information(b) What is the probability that a particle reaches the upper boundary n before the lower boundary m?
MATH 529 The Boudary Problem The drukard s walk (or boudary problem) is oe of the most famous problems i the theory of radom walks. Oe versio of the problem is described as follows: Suppose a particle
More informationWeek 5-6: The Binomial Coefficients
Wee 5-6: The Biomial Coefficiets March 6, 2018 1 Pascal Formula Theorem 11 (Pascal s Formula For itegers ad such that 1, ( ( ( 1 1 + 1 The umbers ( 2 ( 1 2 ( 2 are triagle umbers, that is, The petago umbers
More information, then cv V. Differential Equations Elements of Lineaer Algebra Name: Consider the differential equation. and y2 cos( kx)
Cosider the differetial equatio y '' k y 0 has particular solutios y1 si( kx) ad y cos( kx) I geeral, ay liear combiatio of y1 ad y, cy 1 1 cy where c1, c is also a solutio to the equatio above The reaso
More information