Trustworthy, Useful Languages for. Probabilistic Modeling and Inference
|
|
|
- Amice Mason
- 5 years ago
- Views:
Transcription
1 Trustworthy, Useful Languages for Probabilistic Modeling and Inference Neil Toronto Dissertation Defense Brigham Young University 2014/06/11
2 Master s Research: Super-Resolution Toronto et al. Super-Resolution via Recapture and Bayesian Effect Modeling. CVPR
3 Master s Research: Super-Resolution Model and query: Half a page of beautiful math 2
4 Master s Research: Super-Resolution Query implementation: 600 lines of Python 2
5 Main Results: Super-Resolution Competitor and BEI on 4x super-resolution: Resolution Synthesis 3
6 Main Results: Super-Resolution Competitor and BEI on 4x super-resolution: Resolution Synthesis Bayesian Edge Inference 3
7 Main Results: Super-Resolution Competitor and BEI on 4x super-resolution: Resolution Synthesis Bayesian Edge Inference Beat state-of-the-art on objective measures 3
8 Main Results: Super-Resolution Competitor and BEI on 4x super-resolution: Resolution Synthesis Bayesian Edge Inference Beat state-of-the-art on objective measures Was capable of other reconstruction tasks with few changes 3
9 Only Mostly Satisfying Problem 1: Still not sure the program is right 4
10 Only Mostly Satisfying Problem 1: Still not sure the program is right Problem 2: smooth edges instead of discontinuous 4
11 Only Mostly Satisfying Problem 1: Still not sure the program is right Problem 2: smooth edges instead of discontinuous To approximate blurring with a spatially varying point-spread function (PSF), we assign each facet a Gaussian PSF and convolve each analytically before combining outputs. 4
12 Only Mostly Satisfying Problem 1: Still not sure the program is right Problem 2: smooth edges instead of discontinuous To approximate blurring with a spatially varying point-spread function (PSF), we assign each facet a Gaussian PSF and convolve each analytically before combining outputs. i.e. We can t model it correctly so here s a hack. 4
13 Solution Idea: Probabilistic Language 5
14 Solution Idea: Probabilistic Language Also somehow let me model correctly 5
15 Prior Work Defined by an implementation Defined by a semantics (i.e. mathematically) 6
16 Prior Work Defined by an implementation Defined by a semantics (i.e. mathematically) Designed for Bayesian practice 6
17 Prior Work Defined by an implementation Defined by a semantics (i.e. mathematically) Designed for Bayesian practice Mimic human translation 6
18 Prior Work Defined by an implementation Defined by a semantics (i.e. mathematically) Designed for Bayesian practice Mimic human translation Can t tell error from feature 6
19 Prior Work Defined by an implementation Defined by a semantics (i.e. mathematically) Designed for Bayesian practice Mimic human translation Can t tell error from feature Limited: usually no recursion or loops; conditions 6
20 Prior Work Defined by an implementation Designed for Bayesian practice Defined by a semantics (i.e. mathematically) Designed for functional programmers or FP theorists Mimic human translation Can t tell error from feature Limited: usually no recursion or loops; conditions 6
21 Prior Work Defined by an implementation Designed for Bayesian practice Mimic human translation Defined by a semantics (i.e. mathematically) Designed for functional programmers or FP theorists May not be implemented Can t tell error from feature Limited: usually no recursion or loops; conditions 6
22 Prior Work Defined by an implementation Designed for Bayesian practice Mimic human translation Can t tell error from feature Defined by a semantics (i.e. mathematically) Designed for functional programmers or FP theorists May not be implemented Behavior is well-defined Limited: usually no recursion or loops; conditions 6
23 Prior Work Defined by an implementation Designed for Bayesian practice Mimic human translation Can t tell error from feature Limited: usually no recursion or loops; conditions Defined by a semantics (i.e. mathematically) Designed for functional programmers or FP theorists May not be implemented Behavior is well-defined Limited: usually finite distributions, no conditioning 6
24 Prior Work Defined by an implementation Designed for Bayesian practice Mimic human translation Can t tell error from feature Limited: usually no recursion or loops; conditions Defined by a semantics (i.e. mathematically) Designed for functional programmers or FP theorists May not be implemented Behavior is well-defined Limited: usually finite distributions, no conditioning Best of all worlds: define language using functional programming theory, make it for Bayesians, and remove limitations 6
25 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. 7
26 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. Useful: let you think abstractly and handle details for you 7
27 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. Useful: let you think abstractly and handle details for you Trustworthy: defined mathematically 7
28 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. Useful: let you think abstractly and handle details for you Trustworthy: defined mathematically Functional programming theory has the tools to define programming languages mathematically 7
29 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. Useful: let you think abstractly and handle details for you Trustworthy: defined mathematically Functional programming theory has the tools to define programming languages mathematically Measure-theoretic probability is the most complete account of probability; should allow shedding common limitations 7
30 Simple Example Process Example process: Normal-Normal 8
31 Simple Example Process Example process: Normal-Normal Intuition: Sample, then sample using 8
32 Simple Example Process Example process: Normal-Normal Intuition: Sample, then sample using Density model : 8
33 Simple Example Process Example process: Normal-Normal Intuition: Sample, then sample using Compute query by integrating: 8
34 Conditional Queries Compute query using Bayes law: 9
35 Conditional Queries Compute query using Bayes law: 9
36 Conditional Queries Compute query using Bayes law: 9
37 Conditional Queries Compute query using Bayes law: 9
38 Conditional Queries Compute query using Bayes law: 9
39 Conditional Queries Compute query using Bayes law: 9
40 So What Can t Densities Model? 10
41 So What Can t Densities Model? Tons of useful things that are easy to write down 10
42 So What Can t Densities Model? Tons of useful things that are easy to write down Distributions given non-axial, zero-probability conditions 10
43 So What Can t Densities Model? Tons of useful things that are easy to write down Distributions given non-axial, zero-probability conditions Discontinuous change of variable (e.g. a thermometer) 10
44 So What Can t Densities Model? Tons of useful things that are easy to write down Distributions given non-axial, zero-probability conditions Discontinuous change of variable (e.g. a thermometer) Distributions of variable-dimension random variables 10
45 So What Can t Densities Model? Tons of useful things that are easy to write down Distributions given non-axial, zero-probability conditions Discontinuous change of variable (e.g. a thermometer) Distributions of variable-dimension random variables Nontrivial distributions on infinite products 10
46 So What Can t Densities Model? Tons of useful things that are easy to write down Distributions given non-axial, zero-probability conditions Discontinuous change of variable (e.g. a thermometer) Distributions of variable-dimension random variables Nontrivial distributions on infinite products Tricks to get around limitations aren t general enough 10
47 Measure-Theoretic Probability Main ideas: Don t assign probability-like quantities to values, assign probabilities to sets the probability query is king 11
48 Measure-Theoretic Probability Main ideas: Don t assign probability-like quantities to values, assign probabilities to sets the probability query is king Confine assumed randomness to one place by making random variables deterministic functions that observe a random source 11
49 Measure-Theoretic Probability Main ideas: Don t assign probability-like quantities to values, assign probabilities to sets the probability query is king Confine assumed randomness to one place by making random variables deterministic functions that observe a random source Measure-theoretic model of example process: 11
50 Measure-Theoretic Probability Main ideas: Don t assign probability-like quantities to values, assign probabilities to sets the probability query is king Confine assumed randomness to one place by making random variables deterministic functions that observe a random source Measure-theoretic model of example process: 11
51 Measure-Theoretic Queries Specific query: 12
52 Measure-Theoretic Queries Specific query: Generalized: 12
53 Measure-Theoretic Queries Specific query: Generalized: Conditional query: if then 12
54 Measure-Theoretic Queries Specific query: Generalized: Conditional query: if then Can we avoid densities when? 12
55 Zero-Probability Conditions (Axial) 13
56 Zero-Probability Conditions (Axial) 13
57 Zero-Probability Conditions (Axial) 13
58 Zero-Probability Conditions (Axial) 13
59 Zero-Probability Conditions (Axial) 13
60 Zero-Probability Conditions (Axial) 13
61 Zero-Probability Conditions (Axial) 13
62 Zero-Probability Conditions (Circular) 14
63 Zero-Probability Conditions (Circular) 14
64 Zero-Probability Conditions (Circular) 14
65 Zero-Probability Conditions (Circular) 14
66 Zero-Probability Conditions (Circular) 14
67 Contribution: Don t Integrate, Compute Backwards (1) Integration is hard! 15
68 Contribution: Don t Integrate, Compute Backwards (1) Integration is hard! But random variables and are an abstraction boundary hiding and, so we can choose convenient ones 15
69 Contribution: Don t Integrate, Compute Backwards (1) Integration is hard! But random variables and are an abstraction boundary hiding and, so we can choose convenient ones A uniform random source model: 15
70 Contribution: Don t Integrate, Compute Backwards (1) Integration is hard! But random variables and are an abstraction boundary hiding and, so we can choose convenient ones A uniform random source model: where is the Normal CDF 15
71 Contribution: Don t Integrate, Compute Backwards (1) Integration is hard! But random variables and are an abstraction boundary hiding and, so we can choose convenient ones A uniform random source model: where is the Normal CDF Stretches instead of integrates 15
72 Contribution: Don t Integrate, Compute Backwards (2) Generalized query: 16
73 Contribution: Don t Integrate, Compute Backwards (2) Generalized query: i.e. output distributions are defined by preimages 16
74 Contribution: Don t Integrate, Compute Backwards (2) Generalized query: i.e. output distributions are defined by preimages For a uniform random source model, Compute probabilities by computing preimage areas 16
75 Contribution: Don t Integrate, Compute Backwards (2) Generalized query: i.e. output distributions are defined by preimages For a uniform random source model, Compute probabilities by computing preimage areas Compute conditional probabilities as quotients of preimage areas 16
76 Contribution: Don t Integrate, Compute Backwards (2) Generalized query: i.e. output distributions are defined by preimages For a uniform random source model, Compute probabilities by computing preimage areas Compute conditional probabilities as quotients of preimage areas Is this really more feasible than integrating? 16
77 Queries Using Preimages 17
78 Queries Using Preimages Uniform Random Source Original Model 17
79 Queries Using Preimages Uniform Random Source Original Model 17
80 Queries Using Preimages Uniform Random Source Original Model 17
81 Queries Using Preimages Uniform Random Source Original Model 17
82 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source 18
83 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets 18
84 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets 18
85 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets 18
86 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation 18
87 Crazy Idea is Feasible If... Seems like we need: Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation Completely infeasible! But... 18
88 What About Approximating? Conservative approximation with rectangles: 19
89 What About Approximating? Conservative approximation with rectangles: 19
90 What About Approximating? Restricting preimages to rectangular subdomains: 20
91 What About Approximating? Restricting preimages to rectangular subdomains: 20
92 What About Approximating? Restricting preimages to rectangular subdomains: 20
93 What About Approximating? Restricting preimages to rectangular subdomains: 20
94 What About Approximating? Restricting preimages to rectangular subdomains: 20
95 What About Approximating? Restricting preimages to rectangular subdomains: 20
96 What About Approximating? Restricting preimages to rectangular subdomains: 20
97 What About Approximating? Restricting preimages to rectangular subdomains: 20
98 What About Approximating? Restricting preimages to rectangular subdomains: 20
99 What About Approximating? Restricting preimages to rectangular subdomains: 20
100 What About Approximating? Restricting preimages to rectangular subdomains: 20
101 What About Approximating? Restricting preimages to rectangular subdomains: 20
102 What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 21
103 What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 21
104 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute preimage sets Efficient representation of arbitrary sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 22
105 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute approximate preimage subsets Efficient representation of arbitrary sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 22
106 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute approximate preimage subsets Efficient representation of approximating sets Efficient way to compute volumes of preimage sets Proof of correctness w.r.t. standard interpretation 22
107 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute approximate preimage subsets Efficient representation of approximating sets Efficient way to sample uniformly in preimage sets Proof of correctness w.r.t. standard interpretation 22
108 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute approximate preimage subsets Efficient representation of approximating sets Efficient way to sample uniformly in preimage sets Efficient domain partition sampling Proof of correctness w.r.t. standard interpretation 22
109 Crazy Idea is Actually Feasible If... Standard interpretation of programs as pure functions from a random source Efficient way to compute approximate preimage subsets Efficient representation of approximating sets Efficient way to sample uniformly in preimage sets Efficient domain partition sampling Efficient way to determine whether a domain sample is actually in the preimage (just use standard interpretation) Proof of correctness w.r.t. standard interpretation 22
110 Standard Interpretation Grammar: 23
111 Standard Interpretation Grammar: Semantic function 23
112 Standard Interpretation Grammar: Semantic function Math has no general recursion, so program ) is a λ-calculus term (i.e. interpretation of 23
113 Standard Interpretation Grammar: Semantic function Math has no general recursion, so program ) is a λ-calculus term (i.e. interpretation of Easy implementation in any language with lambdas 23
114 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings 24
115 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction 24
116 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction Example: meaning of 24
117 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction Example: meaning of 24
118 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction Example: meaning of Nonexample: 24
119 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction Example: meaning of Nonexample: 24
120 Compositional Semantics Compositional: every term s meaning depends only on its immediate subterms meanings Advantage: proofs about all programs by structural induction Example: meaning of Nonexample: Can preimages be computed compositionally? 24
121 Pair Preimages 25
122 Pair Preimages 25
123 Pair Preimages : 25
124 Pair Preimages and : 25
125 Pair Preimages : 25
126 Nonstandard Interpretation: Computing Preimages Preimage computation: 26
127 Nonstandard Interpretation: Computing Preimages Preimage computation: 26
128 Nonstandard Interpretation: Preimages Under Pairing Pairing types: 27
129 Nonstandard Interpretation: Preimages Under Pairing Pairing types: Theorem (correctness under pairing). If computes preimages under 27
130 Nonstandard Interpretation: Preimages Under Pairing Pairing types: Theorem (correctness under pairing). If computes preimages under computes preimages under 27
131 Nonstandard Interpretation: Preimages Under Pairing Pairing types: Theorem (correctness under pairing). If computes preimages under computes preimages under then computes preimages under. 27
132 Nonstandard Interpretation: Preimages Under Pairing Pairing types: Theorem (correctness under pairing). If computes preimages under computes preimages under then computes preimages under. Proof sketch. Preimages distribute over cartesian products. 27
133 Nonstandard Interpretation: Preimages Under Pairing Pairing types: Theorem (correctness under pairing). If computes preimages under computes preimages under then computes preimages under. Proof sketch. Preimages distribute over cartesian products. Similar theorems for every kind of term 27
134 Nonstandard Interpretation: Correctness Theorem. For all programs,. computes preimages under Proof. By structural induction on program terms. 28
135 Wait a Minute Q. Don t the interpretations of do uncountable things? 29
136 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. 29
137 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. Q. Where do I get a computer that runs them? 29
138 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. Q. Where do I get a computer that runs them? A. Nowhere, but we ll approximate them soon. 29
139 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. Q. Where do I get a computer that runs them? A. Nowhere, but we ll approximate them soon. Q. Why interpret programs as uncomputable functions, then? 29
140 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. Q. Where do I get a computer that runs them? A. Nowhere, but we ll approximate them soon. Q. Why interpret programs as uncomputable functions, then? A. So we know exactly what to approximate. 29
141 Wait a Minute Q. Don t the interpretations of do uncountable things? A. Yes. Yes, they do. Q. Where do I get a computer that runs them? A. Nowhere, but we ll approximate them soon. Q. Why interpret programs as uncomputable functions, then? A. So we know exactly what to approximate. Q. Where did you get a λ-calculus that could operate on arbitrary, possibly infinite sets, anyway? A. Well... 29
142 Lambda-ZFC λ calculus 30
143 Lambda-ZFC + λ calculus Infinite sets and operations 30
144 Lambda-ZFC + = λ calculus Infinite sets and operations λ ZFC 30
145 Lambda-ZFC + = λ calculus Infinite sets and operations λ ZFC Contemporary math, but with lambdas and general recursion; or functional programming, but with infinite sets 30
146 Lambda-ZFC + = λ calculus Infinite sets and operations λ ZFC Contemporary math, but with lambdas and general recursion; or functional programming, but with infinite sets Can express uncountably infinite operations, can t solve its own halting problem 30
147 Lambda-ZFC + = λ calculus Infinite sets and operations λ ZFC Contemporary math, but with lambdas and general recursion; or functional programming, but with infinite sets Can express uncountably infinite operations, can t solve its own halting problem Can use contemporary mathematical theorems directly 30
148 Rectangular Approximation A rectangle is An interval or union of intervals for rectangles and 31
149 Rectangular Approximation A rectangle is An interval or union of intervals for rectangles and Easy representation; easy intersection and join (union-like) operation, empty test, other operations 31
150 Rectangular Approximation A rectangle is An interval or union of intervals for rectangles and Easy representation; easy intersection and join (union-like) operation, empty test, other operations Recall: 31
151 Rectangular Approximation A rectangle is An interval or union of intervals for rectangles and Easy representation; easy intersection and join (union-like) operation, empty test, other operations Recall: Define: 31
152 Rectangular Approximation A rectangle is An interval or union of intervals for rectangles and Easy representation; easy intersection and join (union-like) operation, empty test, other operations Recall: Define: Derive 31
153 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling within preimages doesn t leave anything out 32
154 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling within preimages doesn t leave anything out Theorem (monotone). is monotone. Consequence: Partitioning the domain never increases approximate preimages 32
155 In Theory... Theorem (sound). computes overapproximations of the preimages computed by. Consequence: Sampling within preimages doesn t leave anything out Theorem (monotone). is monotone. Consequence: Partitioning the domain never increases approximate preimages Theorem (decreasing). the given subdomain. never returns preimages larger than Consequence: Refining preimage partitions never explodes 32
156 In Practice... Theorems prove this always works: 33
157 In Practice... Theorems prove this always works: 33
158 In Practice... Theorems prove this always works: 33
159 In Practice... Theorems prove this always works: 33
160 In Practice... Theorems prove this always works: 33
161 In Practice... Theorems prove this always works: 33
162 In Practice... Theorems prove this always works: 33
163 In Practice... Theorems prove this always works: 33
164 In Practice... Theorems prove this always works: 33
165 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: 34
166 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: First, refine using preimage computation: 34
167 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: Second, randomly choose from arbitrarily fine partition: 34
168 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: Third, refine again: 34
169 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: Fourth, sample uniformly: 34
170 Importance Sampling Alternative to arbitrarily low-rate rejection sampling: Do process in the limit ; i.e. choose : 34
171 What About Recursion? General recursion, programs that halt with probability 1; e.g. (define/drbayes (geometric p) (if (bernoulli p) 0 (+ 1 (geometric p)))) 35
172 What About Recursion? General recursion, programs that halt with probability 1; e.g. (define/drbayes (geometric p) (if (bernoulli p) 0 (+ 1 (geometric p)))) Consider programs as being fully inlined (thus infinite): (if (bernoulli p) 0 (+ 1 (if (bernoulli p) 0 (+ 1 (if (bernoulli p) 0 (+ 1...)))))) 35
173 What About Recursion? General recursion, programs that halt with probability 1; e.g. (define/drbayes (geometric p) (if (bernoulli p) 0 (+ 1 (geometric p)))) Consider programs as being fully inlined (thus infinite): (if (bernoulli p) 0 (+ 1 (if (bernoulli p) 0 (+ 1 (if (bernoulli p) 0 (+ 1...)))))) Random domain needs to be big enough and the right shape 35
174 Program Domain Values Values are infinite binary trees: 36
175 Program Domain Values Values are infinite binary trees: Every expression in a program is assigned a node 36
176 Program Domain Values Values are infinite binary trees: Every expression in a program is assigned a node Implemented using lazy trees of random values 36
177 Program Domain Values Values are infinite binary trees: Every expression in a program is assigned a node Implemented using lazy trees of random values No probability density for domain, but there is a measure 36 36
178 Demo: Normal-Normal With Circular Condition Normal-Normal process: 37
179 Demo: Normal-Normal With Circular Condition Normal-Normal process: Objective: Find the distribution of 37
180 Demo: Normal-Normal With Circular Condition Normal-Normal process: Objective: Find the distribution of Implementation: (define/drbayes e (let* ([x (normal 0 1)] [y (normal x 1)]) (list x y (sqrt (+ (sqr x) (sqr y)))))) 37
181 Demo: Normal-Normal With Circular Condition Normal-Normal process: Objective: Find the distribution of Implementation: (define/drbayes e (let* ([x (normal 0 1)] [y (normal x 1)]) (list x y (sqrt (+ (sqr x) (sqr y)))))) Goal: Sample in the preimage of (set-list reals reals (interval (- 1 ε) (+ 1 ε))) 37
182 Demo: Normal-Normal With Circular Condition For ε = 0.01: Preimage rectangles 38
183 Demo: Normal-Normal With Circular Condition For ε = 0.01: Preimage samples 38
184 Demo: Normal-Normal With Circular Condition For ε = 0.01: Preimage samples Output samples 38
185 Demo: Normal-Normal With Circular Condition For ε = 0.01: Preimage samples Output samples Works fine with much smaller ε 38
186 Demo: Thermometer Normal-Normal thermometer process: 39
187 Demo: Thermometer Normal-Normal thermometer process: Objective: Find the distribution of 39
188 Demo: Thermometer Normal-Normal thermometer process: Objective: Find the distribution of Implementation: (define/drbayes e (let* ([x (normal 90 10)] [y (normal x 1)]) (list x (if (> y 100) 100 y)))) 39
189 Demo: Thermometer Normal-Normal thermometer process: Objective: Find the distribution of Implementation: (define/drbayes e (let* ([x (normal 90 10)] [y (normal x 1)]) (list x (if (> y 100) 100 y)))) Goal: Sample in the preimage of (set-list reals (interval )) 39 39
190 Demo: Thermometer Preimage rectangles 40
191 Demo: Thermometer Preimage samples 40
192 Demo: Thermometer Preimage samples Density of 40
193 Demo: Thermometer Preimage samples Density of Calculated from samples: mean 105.1, stddev
194 Demo: Stochastic Ray Tracing Idea: Model light transmission and reflection, condition on paths that pass through aperture 41
195 Demo: Stochastic Ray Tracing Idea: Model light transmission and reflection, condition on paths that pass through aperture 41
196 Demo: Stochastic Ray Tracing Part of the implementation (totals ~50 lines): (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (vec-dot v n))]) (if (positive? denom) (let ([t (/ (+ d (vec-dot p0 n)) denom)]) (if (positive? t) (collision t (vec+ p0 (vec-scale v t)) n) #f)) #f))) 42
![Demo: Stochastic Ray Tracing Part of the implementation (totals ~50 lines): (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (vec-dot v n))]) (if (positive?](/docs-images/85/92059994/images/197-2.jpg "denom) (let ([t (/ (+ d (vec-dot p0 n)) denom)]) (if (positive?")
197 Demo: Stochastic Ray Tracing Part of the implementation (totals ~50 lines): (define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (vec-dot v n))]) (if (positive? denom) (let ([t (/ (+ d (vec-dot p0 n)) denom)]) (if (positive? t) (collision t (vec+ p0 (vec-scale v t)) n) #f)) #f))) Constrained light path outputs: Paths Through Aperture Projected and Accumulated 42
198 Other Inference Tasks Typical Hierarchical models Bayesian regression Model selection 43
199 Other Inference Tasks Typical Hierarchical models Bayesian regression Model selection Atypical Programs that halt with probability < 1, or never halt Probabilistic program verification (sample in preimage of error condition) 43
200 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. 44
201 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. True. 44
202 Thesis Statement Functional programming theory and measure-theoretic probability provide a solid foundation for trustworthy, useful languages for constructive probabilistic modeling and inference. True. Was it falsifiable? 44
203 Measurability Only measurable sets can have probabilities 45
204 Measurability Only measurable sets can have probabilities Computing preimages under say itself is measurable must preserve measurability we 45
205 Measurability Only measurable sets can have probabilities Computing preimages under say itself is measurable must preserve measurability we Theorem (measurability). For all programs, is measurable, regardless of errors or nontermination, if language primitives are measurable. 45
206 Measurability Only measurable sets can have probabilities Computing preimages under say itself is measurable must preserve measurability we Theorem (measurability). For all programs, is measurable, regardless of errors or nontermination, if language primitives are measurable. Primitives include uncomputable operations like limits 45
207 Measurability Only measurable sets can have probabilities Computing preimages under say itself is measurable must preserve measurability we Theorem (measurability). For all programs, is measurable, regardless of errors or nontermination, if language primitives are measurable. Primitives include uncomputable operations like limits Applies to all probabilistic programming languages 45
208 What I Did 45
209 What I Did The core calculus for this: 45
210 Future Work Expressiveness Lambdas and macros Exceptions, parameters (or continuations and marks) 46
211 Future Work Expressiveness Lambdas and macros Exceptions, parameters (or continuations and marks) Optimization Direct implementation is in depth; cut to Incremental computation Adaptive sampling algorithms Static analysis 46
212 Future Work Expressiveness Lambdas and macros Exceptions, parameters (or continuations and marks) Optimization Direct implementation is in depth; cut to Incremental computation Adaptive sampling algorithms Static analysis Branching out: investigate preimage computation connection with type systems and predicate transformer semantics 46
CSE 311: Foundations of Computing I Autumn 2014 Practice Final: Section X. Closed book, closed notes, no cell phones, no calculators.
CSE 311: Foundations of Computing I Autumn 014 Practice Final: Section X YY ZZ Name: UW ID: Instructions: Closed book, closed notes, no cell phones, no calculators. You have 110 minutes to complete the
15-251: Great Theoretical Ideas in Computer Science Lecture 7. Turing s Legacy Continues
15-251: Great Theoretical Ideas in Computer Science Lecture 7 Turing s Legacy Continues Solvable with Python = Solvable with C = Solvable with Java = Solvable with SML = Decidable Languages (decidable
CS 301. Lecture 18 Decidable languages. Stephen Checkoway. April 2, 2018
CS 301 Lecture 18 Decidable languages Stephen Checkoway April 2, 2018 1 / 26 Decidable language Recall, a language A is decidable if there is some TM M that 1 recognizes A (i.e., L(M) = A), and 2 halts
Extensions to the Logic of All x are y: Verbs, Relative Clauses, and Only
1/53 Extensions to the Logic of All x are y: Verbs, Relative Clauses, and Only Larry Moss Indiana University Nordic Logic School August 7-11, 2017 2/53 An example that we ll see a few times Consider the
18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
Math 541 Fall 2008 Connectivity Transition from Math 453/503 to Math 541 Ross E. Staffeldt-August 2008
Math 541 Fall 2008 Connectivity Transition from Math 453/503 to Math 541 Ross E. Staffeldt-August 2008 Closed sets We have been operating at a fundamental level at which a topological space is a set together
Nondeterministic finite automata
Lecture 3 Nondeterministic finite automata This lecture is focused on the nondeterministic finite automata (NFA) model and its relationship to the DFA model. Nondeterminism is an important concept in the
1 Reals are Uncountable
CS 30: Discrete Math in CS (Winter 2019): Lecture 6 Date: 11th January, 2019 (Friday) Topic: Uncountability and Undecidability Disclaimer: These notes have not gone through scrutiny and in all probability
T (s, xa) = T (T (s, x), a). The language recognized by M, denoted L(M), is the set of strings accepted by M. That is,
Recall A deterministic finite automaton is a five-tuple where S is a finite set of states, M = (S, Σ, T, s 0, F ) Σ is an alphabet the input alphabet, T : S Σ S is the transition function, s 0 S is the
We are going to discuss what it means for a sequence to converge in three stages: First, we define what it means for a sequence to converge to zero
Chapter Limits of Sequences Calculus Student: lim s n = 0 means the s n are getting closer and closer to zero but never gets there. Instructor: ARGHHHHH! Exercise. Think of a better response for the instructor.
Propositional and Predicate Logic - V
Propositional and Predicate Logic - V Petr Gregor KTIML MFF UK WS 2016/2017 Petr Gregor (KTIML MFF UK) Propositional and Predicate Logic - V WS 2016/2017 1 / 21 Formal proof systems Hilbert s calculus
CSE 311 Lecture 28: Undecidability of the Halting Problem. Emina Torlak and Kevin Zatloukal
CSE 311 Lecture 28: Undecidability of the Halting Problem Emina Torlak and Kevin Zatloukal 1 Topics Final exam Logistics, format, and topics. Countability and uncomputability A quick recap of Lecture 27.
Linear Classifiers and the Perceptron
Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers Let s assume that every instance is an n-dimensional vector of real numbers x R n, and there are only two possible
19. TAYLOR SERIES AND TECHNIQUES
19. TAYLOR SERIES AND TECHNIQUES Taylor polynomials can be generated for a given function through a certain linear combination of its derivatives. The idea is that we can approximate a function by a polynomial,
Further discussion of Turing machines
Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will
Complexity Theory Part II
Complexity Theory Part II Time Complexity The time complexity of a TM M is a function denoting the worst-case number of steps M takes on any input of length n. By convention, n denotes the length of the
Lecture Notes on Inductive Definitions
Lecture Notes on Inductive Definitions 15-312: Foundations of Programming Languages Frank Pfenning Lecture 2 September 2, 2004 These supplementary notes review the notion of an inductive definition and
dynamical zeta functions: what, why and what are the good for?
dynamical zeta functions: what, why and what are the good for? Predrag Cvitanović Georgia Institute of Technology November 2 2011 life is intractable in physics, no problem is tractable I accept chaos
COMPLEX NUMBERS II L. MARIZZA A. BAILEY
COMPLEX NUMBERS II L. MARIZZA A. BAILEY 1. Roots of Unity Previously, we learned that the product of z = z cis(θ), w = w cis(α) C is represented geometrically by the rotation of z by the angle α = arg
Models of Computation,
Models of Computation, 2010 1 Induction We use a lot of inductive techniques in this course, both to give definitions and to prove facts about our semantics So, it s worth taking a little while to set
Theory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
Tips and Tricks in Real Analysis
Tips and Tricks in Real Analysis Nate Eldredge August 3, 2008 This is a list of tricks and standard approaches that are often helpful when solving qual-type problems in real analysis. Approximate. There
Random variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
Analysis of Algorithms I: Perfect Hashing
Analysis of Algorithms I: Perfect Hashing Xi Chen Columbia University Goal: Let U = {0, 1,..., p 1} be a huge universe set. Given a static subset V U of n keys (here static means we will never change the
Safety Analysis versus Type Inference
Information and Computation, 118(1):128 141, 1995. Safety Analysis versus Type Inference Jens Palsberg palsberg@daimi.aau.dk Michael I. Schwartzbach mis@daimi.aau.dk Computer Science Department, Aarhus
Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018
CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of
Simulation. Where real stuff starts
1 Simulation Where real stuff starts ToC 1. What is a simulation? 2. Accuracy of output 3. Random Number Generators 4. How to sample 5. Monte Carlo 6. Bootstrap 2 1. What is a simulation? 3 What is a simulation?
6.045: Automata, Computability, and Complexity (GITCS) Class 17 Nancy Lynch
6.045: Automata, Computability, and Complexity (GITCS) Class 17 Nancy Lynch Today Probabilistic Turing Machines and Probabilistic Time Complexity Classes Now add a new capability to standard TMs: random
Turing Machines Part Three
Turing Machines Part Three What problems can we solve with a computer? What kind of computer? Very Important Terminology Let M be a Turing machine. M accepts a string w if it enters an accept state when
Data Structures for Efficient Inference and Optimization
Data Structures for Efficient Inference and Optimization in Expressive Continuous Domains Scott Sanner Ehsan Abbasnejad Zahra Zamani Karina Valdivia Delgado Leliane Nunes de Barros Cheng Fang Discrete
Compute the Fourier transform on the first register to get x {0,1} n x 0.
CS 94 Recursive Fourier Sampling, Simon s Algorithm /5/009 Spring 009 Lecture 3 1 Review Recall that we can write any classical circuit x f(x) as a reversible circuit R f. We can view R f as a unitary
THEORY OF COMPUTATION (AUBER) EXAM CRIB SHEET
THEORY OF COMPUTATION (AUBER) EXAM CRIB SHEET Regular Languages and FA A language is a set of strings over a finite alphabet Σ. All languages are finite or countably infinite. The set of all languages
CPSC 421: Tutorial #1
CPSC 421: Tutorial #1 October 14, 2016 Set Theory. 1. Let A be an arbitrary set, and let B = {x A : x / x}. That is, B contains all sets in A that do not contain themselves: For all y, ( ) y B if and only
MT101-03, Area Notes. We briefly mentioned the following triumvirate of integration that we ll be studying this semester:
We briefly mentioned the following triumvirate of integration that we ll be studying this semester: AD AUG MT101-03, Area Notes NC where AUG stands for Area Under the Graph of a positive elementary function,
Most General computer?
Turing Machines Most General computer? DFAs are simple model of computation. Accept only the regular languages. Is there a kind of computer that can accept any language, or compute any function? Recall
Computational and Statistical Learning theory
Computational and Statistical Learning theory Problem set 2 Due: January 31st Email solutions to : karthik at ttic dot edu Notation : Input space : X Label space : Y = {±1} Sample : (x 1, y 1,..., (x n,
From Constructibility and Absoluteness to Computability and Domain Independence
From Constructibility and Absoluteness to Computability and Domain Independence Arnon Avron School of Computer Science Tel Aviv University, Tel Aviv 69978, Israel aa@math.tau.ac.il Abstract. Gödel s main
Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16
600.463 Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16 25.1 Introduction Today we re going to talk about machine learning, but from an
1 Trees. Listing 1: Node with two child reference. public class ptwochildnode { protected Object data ; protected ptwochildnode l e f t, r i g h t ;
1 Trees The next major set of data structures belongs to what s called Trees. They are called that, because if you try to visualize the structure, it kind of looks like a tree (root, branches, and leafs).
Lecture Notes on Inductive Definitions
Lecture Notes on Inductive Definitions 15-312: Foundations of Programming Languages Frank Pfenning Lecture 2 August 28, 2003 These supplementary notes review the notion of an inductive definition and give
A Stochastic l-calculus
A Stochastic l-calculus Content Areas: probabilistic reasoning, knowledge representation, causality Tracking Number: 775 Abstract There is an increasing interest within the research community in the design
Math 111, Introduction to the Calculus, Fall 2011 Midterm I Practice Exam 1 Solutions
Math 111, Introduction to the Calculus, Fall 2011 Midterm I Practice Exam 1 Solutions For each question, there is a model solution (showing you the level of detail I expect on the exam) and then below
Undecidable Problems. Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, / 65
Undecidable Problems Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, 2018 1/ 65 Algorithmically Solvable Problems Let us assume we have a problem P. If there is an algorithm solving
NP-Completeness and Boolean Satisfiability
NP-Completeness and Boolean Satisfiability Mridul Aanjaneya Stanford University August 14, 2012 Mridul Aanjaneya Automata Theory 1/ 49 Time-Bounded Turing Machines A Turing Machine that, given an input
Thus, X is connected by Problem 4. Case 3: X = (a, b]. This case is analogous to Case 2. Case 4: X = (a, b). Choose ε < b a
![Thus, X is connected by Problem 4. Case 3: X = (a, b]. This case is analogous to Case 2. Case 4: X = (a, b). Choose ε < b a](/thumbs/93/112528193.jpg "Thus, X is connected by Problem 4. Case 3: X = (a, b]. This case is analogous to Case 2. Case 4: X = (a, b). Choose ε < b a")
Solutions to Homework #6 1. Complete the proof of the backwards direction of Theorem 12.2 from class (which asserts the any interval in R is connected). Solution: Let X R be a closed interval. Case 1:
CA320 - Computability & Complexity
CA320 - Computability & Complexity David Sinclair Overview In this module we are going to answer 2 important questions: Can all problems be solved by a computer? What problems be efficiently solved by
CS357: CTL Model Checking (two lectures worth) David Dill
CS357: CTL Model Checking (two lectures worth) David Dill 1 CTL CTL = Computation Tree Logic It is a propositional temporal logic temporal logic extended to properties of events over time. CTL is a branching
Undecidability. Andreas Klappenecker. [based on slides by Prof. Welch]
![Undecidability. Andreas Klappenecker. [based on slides by Prof. Welch]](/thumbs/81/83378912.jpg "Undecidability. Andreas Klappenecker. [based on slides by Prof. Welch]")
Undecidability Andreas Klappenecker [based on slides by Prof. Welch] 1 Sources Theory of Computing, A Gentle Introduction, by E. Kinber and C. Smith, Prentice-Hall, 2001 Automata Theory, Languages and
Logic in AI Chapter 7. Mausam (Based on slides of Dan Weld, Stuart Russell, Dieter Fox, Henry Kautz )
Logic in AI Chapter 7 Mausam (Based on slides of Dan Weld, Stuart Russell, Dieter Fox, Henry Kautz ) Knowledge Representation represent knowledge in a manner that facilitates inferencing (i.e. drawing
Generating Function Notes , Fall 2005, Prof. Peter Shor
Counting Change Generating Function Notes 80, Fall 00, Prof Peter Shor In this lecture, I m going to talk about generating functions We ve already seen an example of generating functions Recall when we
Acknowledgments 2. Part 0: Overview 17
Contents Acknowledgments 2 Preface for instructors 11 Which theory course are we talking about?.... 12 The features that might make this book appealing. 13 What s in and what s out............... 14 Possible
Logic in AI Chapter 7. Mausam (Based on slides of Dan Weld, Stuart Russell, Subbarao Kambhampati, Dieter Fox, Henry Kautz )
Logic in AI Chapter 7 Mausam (Based on slides of Dan Weld, Stuart Russell, Subbarao Kambhampati, Dieter Fox, Henry Kautz ) 2 Knowledge Representation represent knowledge about the world in a manner that
Lecture 11: Hash Functions, Merkle-Damgaard, Random Oracle
CS 7880 Graduate Cryptography October 20, 2015 Lecture 11: Hash Functions, Merkle-Damgaard, Random Oracle Lecturer: Daniel Wichs Scribe: Tanay Mehta 1 Topics Covered Review Collision-Resistant Hash Functions
Turing Machine Recap
Turing Machine Recap DFA with (infinite) tape. One move: read, write, move, change state. High-level Points Church-Turing thesis: TMs are the most general computing devices. So far no counter example Every
Math 105A HW 1 Solutions
Sect. 1.1.3: # 2, 3 (Page 7-8 Math 105A HW 1 Solutions 2(a ( Statement: Each positive integers has a unique prime factorization. n N: n = 1 or ( R N, p 1,..., p R P such that n = p 1 p R and ( n, R, S
Theory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
Temporal logics and explicit-state model checking. Pierre Wolper Université de Liège
Temporal logics and explicit-state model checking Pierre Wolper Université de Liège 1 Topics to be covered Introducing explicit-state model checking Finite automata on infinite words Temporal Logics and
Probability Experiments, Trials, Outcomes, Sample Spaces Example 1 Example 2
Probability Probability is the study of uncertain events or outcomes. Games of chance that involve rolling dice or dealing cards are one obvious area of application. However, probability models underlie
Chapter 20. Countability The rationals and the reals. This chapter covers infinite sets and countability.
Chapter 20 Countability This chapter covers infinite sets and countability. 20.1 The rationals and the reals You re familiar with three basic sets of numbers: the integers, the rationals, and the reals.
CS 5114: Theory of Algorithms. Tractable Problems. Tractable Problems (cont) Decision Problems. Clifford A. Shaffer. Spring 2014
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2014 by Clifford A. Shaffer : Theory of Algorithms Title page : Theory of Algorithms Clifford A. Shaffer Spring 2014 Clifford
Takeaway Notes: Finite State Automata
Takeaway Notes: Finite State Automata Contents 1 Introduction 1 2 Basics and Ground Rules 2 2.1 Building Blocks.............................. 2 2.2 The Name of the Game.......................... 2 3 Deterministic
CS154, Lecture 10: Rice s Theorem, Oracle Machines
CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable
Limits of Computation
The real danger is not that computers will begin to think like men, but that men will begin to think like computers Limits of Computation - Sydney J. Harris What makes you believe now that I am just talking
Recap (1) 1. Automata, grammars and other formalisms can be considered as mechanical devices to solve mathematical problems
Computability 1 Recap (1) 1. Automata, grammars and other formalisms can be considered as mechanical devices to solve mathematical problems Mathematical problems are often the formalization of some practical
Decidability (What, stuff is unsolvable?)
University of Georgia Fall 2014 Outline Decidability Decidable Problems for Regular Languages Decidable Problems for Context Free Languages The Halting Problem Countable and Uncountable Sets Diagonalization
One-Parameter Processes, Usually Functions of Time
Chapter 4 One-Parameter Processes, Usually Functions of Time Section 4.1 defines one-parameter processes, and their variations (discrete or continuous parameter, one- or two- sided parameter), including
Conditional probabilities and graphical models
Conditional probabilities and graphical models Thomas Mailund Bioinformatics Research Centre (BiRC), Aarhus University Probability theory allows us to describe uncertainty in the processes we model within
Building Infinite Processes from Finite-Dimensional Distributions
Chapter 2 Building Infinite Processes from Finite-Dimensional Distributions Section 2.1 introduces the finite-dimensional distributions of a stochastic process, and shows how they determine its infinite-dimensional
} Some languages are Turing-decidable A Turing Machine will halt on all inputs (either accepting or rejecting). No infinite loops.
and their languages } Some languages are Turing-decidable A Turing Machine will halt on all inputs (either accepting or rejecting). No infinite loops. } Some languages are Turing-recognizable, but not
Lecture 9: Large Margin Classifiers. Linear Support Vector Machines
Lecture 9: Large Margin Classifiers. Linear Support Vector Machines Perceptrons Definition Perceptron learning rule Convergence Margin & max margin classifiers (Linear) support vector machines Formulation
Propositional Logic. Logic. Propositional Logic Syntax. Propositional Logic
Propositional Logic Reading: Chapter 7.1, 7.3 7.5 [ased on slides from Jerry Zhu, Louis Oliphant and ndrew Moore] Logic If the rules of the world are presented formally, then a decision maker can use logical
Denoting computation
A jog from Scott Domains to Hypercoherence Spaces 13/12/2006 Outline Motivation 1 Motivation 2 What Does Denotational Semantic Mean? Trivial examples Basic things to know 3 Scott domains di-domains 4 Event
CS 188: Artificial Intelligence. Bayes Nets
CS 188: Artificial Intelligence Probabilistic Inference: Enumeration, Variable Elimination, Sampling Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew
Theoretical Cryptography, Lectures 18-20
Theoretical Cryptography, Lectures 18-20 Instructor: Manuel Blum Scribes: Ryan Williams and Yinmeng Zhang March 29, 2006 1 Content of the Lectures These lectures will cover how someone can prove in zero-knowledge
MATH2206 Prob Stat/20.Jan Weekly Review 1-2
MATH2206 Prob Stat/20.Jan.2017 Weekly Review 1-2 This week I explained the idea behind the formula of the well-known statistic standard deviation so that it is clear now why it is a measure of dispersion
Turing s Legacy Continues
15-251: Great Theoretical Ideas in Computer Science Lecture 6 Turing s Legacy Continues Solvable with Python = Solvable with C = Solvable with Java = Solvable with SML = Decidable Languages (decidable
CS2742 midterm test 2 study sheet. Boolean circuits: Predicate logic:
x NOT ~x x y AND x /\ y x y OR x \/ y Figure 1: Types of gates in a digital circuit. CS2742 midterm test 2 study sheet Boolean circuits: Boolean circuits is a generalization of Boolean formulas in which
Why Bayesian? Rigorous approach to address statistical estimation problems. The Bayesian philosophy is mature and powerful.
Why Bayesian? Rigorous approach to address statistical estimation problems. The Bayesian philosophy is mature and powerful. Even if you aren t Bayesian, you can define an uninformative prior and everything
MAT 271E Probability and Statistics
MAT 71E Probability and Statistics Spring 013 Instructor : Class Meets : Office Hours : Textbook : Supp. Text : İlker Bayram EEB 1103 ibayram@itu.edu.tr 13.30 1.30, Wednesday EEB 5303 10.00 1.00, Wednesday
Turing Machines and Time Complexity
Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move
Number Systems. There are 10 kinds of people those that understand binary, those that don t, and those that expected this joke to be in base 2
Number Systems There are 10 kinds of people those that understand binary, those that don t, and those that expected this joke to be in base 2 A Closer Look at the Numbers We Use What is the difference
Mathematics-I Prof. S.K. Ray Department of Mathematics and Statistics Indian Institute of Technology, Kanpur. Lecture 1 Real Numbers
Mathematics-I Prof. S.K. Ray Department of Mathematics and Statistics Indian Institute of Technology, Kanpur Lecture 1 Real Numbers In these lectures, we are going to study a branch of mathematics called
Introduction to Computer Science and Programming for Astronomers
Introduction to Computer Science and Programming for Astronomers Lecture 8. István Szapudi Institute for Astronomy University of Hawaii March 7, 2018 Outline Reminder 1 Reminder 2 3 4 Reminder We have
Math 350: An exploration of HMMs through doodles.
Math 350: An exploration of HMMs through doodles. Joshua Little (407673) 19 December 2012 1 Background 1.1 Hidden Markov models. Markov chains (MCs) work well for modelling discrete-time processes, or
Introduction to Logic and Axiomatic Set Theory
Introduction to Logic and Axiomatic Set Theory 1 Introduction In mathematics, we seek absolute rigor in our arguments, and a solid foundation for all of the structures we consider. Here, we will see some
Introduction to Probabilistic Programming Language (with Church as an example) Presenter: Enrique Rosales, Xing Zeng
Introduction to Probabilistic Programming Language (with Church as an example) Presenter: Enrique Rosales, Xing Zeng 1 Knowledge How can we infer knowledge from observations? 2 Ron s box Bob has a box
CSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?
CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to
Data Flow Analysis. Lecture 6 ECS 240. ECS 240 Data Flow Analysis 1
Data Flow Analysis Lecture 6 ECS 240 ECS 240 Data Flow Analysis 1 The Plan Introduce a few example analyses Generalize to see the underlying theory Discuss some more advanced issues ECS 240 Data Flow Analysis
Math101, Sections 2 and 3, Spring 2008 Review Sheet for Exam #2:
Math101, Sections 2 and 3, Spring 2008 Review Sheet for Exam #2: 03 17 08 3 All about lines 3.1 The Rectangular Coordinate System Know how to plot points in the rectangular coordinate system. Know the
Math 144 Summer 2012 (UCR) Pro-Notes June 24, / 15
Before we start, I want to point out that these notes are not checked for typos. There are prbally many typeos in them and if you find any, please let me know as it s extremely difficult to find them all
Sequence convergence, the weak T-axioms, and first countability
Sequence convergence, the weak T-axioms, and first countability 1 Motivation Up to now we have been mentioning the notion of sequence convergence without actually defining it. So in this section we will
Semantics and Pragmatics of NLP
Semantics and Pragmatics of NLP Alex Ewan School of Informatics University of Edinburgh 28 January 2008 1 2 3 Taking Stock We have: Introduced syntax and semantics for FOL plus lambdas. Represented FOL
CSC236 Week 10. Larry Zhang
CSC236 Week 10 Larry Zhang 1 Today s Topic Deterministic Finite Automata (DFA) 2 Recap of last week We learned a lot of terminologies alphabet string length of string union concatenation Kleene star language
CS1800: Mathematical Induction. Professor Kevin Gold
CS1800: Mathematical Induction Professor Kevin Gold Induction: Used to Prove Patterns Just Keep Going For an algorithm, we may want to prove that it just keeps working, no matter how big the input size
Computability and Complexity
Computability and Complexity Decidability, Undecidability and Reducibility; Codes, Algorithms and Languages CAS 705 Ryszard Janicki Department of Computing and Software McMaster University Hamilton, Ontario,
Turing Machines Part III
Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's
Context-free grammars and languages
Context-free grammars and languages The next class of languages we will study in the course is the class of context-free languages. They are defined by the notion of a context-free grammar, or a CFG for
CSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
2. TRIGONOMETRY 3. COORDINATEGEOMETRY: TWO DIMENSIONS
1 TEACHERS RECRUITMENT BOARD, TRIPURA (TRBT) EDUCATION (SCHOOL) DEPARTMENT, GOVT. OF TRIPURA SYLLABUS: MATHEMATICS (MCQs OF 150 MARKS) SELECTION TEST FOR POST GRADUATE TEACHER(STPGT): 2016 1. ALGEBRA Sets:
CS173 Strong Induction and Functions. Tandy Warnow
CS173 Strong Induction and Functions Tandy Warnow CS 173 Introduction to Strong Induction (also Functions) Tandy Warnow Preview of the class today What are functions? Weak induction Strong induction A