Machine Learning for Large-Scale Data Analysis and Decision Making A. Week #1

Transcription

1 Machine Learning for Large-Scale Data Analysis and Decision Making A Week #1

2 Today Introduction to machine learning The course (syllabus) Math review (probability + linear algebra) The future of machine learning Why are you enrolled and what do you want from this class?

3 Machine Learning (ML) Science that studies statistical and computational aspects of modeling data for predictive purposes (Mostly) Empirical science

4 Data ML Predictions Task: Predict whether an image contains a malignant tumor. Task: Predict the next movie a person should watch.

5

6 Subjective view of how ML relates to other fields

7 Historical View (Modern) Statistics: ~1900 Machine Learning and Data Mining: ~1960 Data Science: ~2000

8 Computer Science + Engineering Machine Learning Statistics + Mathematics Danger Zone Data Science (BI) Traditional Research Substantive Expertise [Inspired by Drew Conway]

9 Data analysis, machine learning and data mining are various names given to the practice of statistical inference, depending on the context. Probability Data generating process Observed data Inference Larry Wasserman in All of Statistics: A Concise Course in Statistical Inference.

10 [Reproduced from R. Neal: ]

11 ML vs. other fields Traditional statistics Causality and understanding are more important than prediction. Data size is less important. Less computational aspects. Simpler, linear relationships. Data mining (somewhat synonymous of ML) Computational Efficiency [ ] more important than statistical sophistication. Often business-related problems. [Ideas and text borrowed from R. Neal: ]

12 Data View Statistics Designs experiments to collect data Machine Learning Uses available data but also designs collection mechanisms Data Science Uses data made available by other processes

13 Artificial Machine Learning Intelligence Deep Learning

14 Applications of ML

15

16

17

18

19

20

21

22 Medicine: personalized, automate diagnostics Social sciences: prediction problem (e.g., predict recidivism) Engineering: to propose new design, evaluate without building Finance: capture uncertainty, short-term trading Marketing: to understand and quantify user experience, advertising efficacy Many others: conservation, social projects Your domain of expertise

23 Course Introduction & Goals

24 Course webpage Google my name. There s a link from my home page.

25 Fit with other courses HEC PhD level Computationally oriented Other ML courses in Montreal (U.Montreal, Polytechnique, Mcgill) More applied Emphasize on large-scale data using distributed computational methods

26 Short review of linear algebra, statistics, and probabilities

27 Based on chapters 2 and 3 of Deep Learning

28 Linear algebra Scalar: a single value. a R, a N a = 3 Vector: an array of values. a R D, a N D a = Matrix: a table of values. A R D 1 D 2, A N D 1 D 2 [ ] A = 1 2 9

29 Indexing notation Indexing elements of a vector: 3 a = 4 2 a 1 a i Convention: The first element is the zero th. Indexing elements of a matrix: a ij [ ] A = a 12

30 Simple operations Transpose a 0 a = a 1 a 2 (A ij ) = A ji a = a 0 a 1 a 2 Addition Vectors and matrices w. the same shape b 0 a 0 + b 0 b = b 1 a + b = b 2 a 1 + b 1 a 2 + b 2 (A + B) ij = A ij + B ij

31 Simple operations Multiply by a scalar a 0 a = a 1 a 2 Vector product. The dot product a a = i a i a i Note: it yields a scalar. Element-wise product: a a = a 0 a 0 a 1 a 1 Also known as Hadamard product a 2 a 2

32 Operations Matrix product (dot product): C ij = k A ik B kj A's columns must equal B s rows (order is important) A R D 1 D 2, B R D 2 D 3 Distributive: A(B + C) =AB + AC Associative: A(BC) =(AB)C (AB) = B A

33 Inverse We denote a matrix s inverse as A 1 A matrix has an inverse iff: it s square. D 1 = D 2 its columns are linearly independent. A square matrix not inversible is singular No column can be recovered using a combination of other columns Inverses are useful to solve systems of equations: Ax = b x = A 1 b

34 Norms norm. Size of a vector (or matrix) L p 1/p a p = a i p i Standard norms in ML: Euclidean norm (p=2) a 2 = a i 2 i Dot product w. 2-norm: a t b = a 2 b 2 cos xy Frobenius norm (matrix): A 2 = i j a ij 2

35 Special matrices & vectors Identity. Denoted I n I 3 = All zeros except for ones on the main diagonal Symmetric: A = A Unit vector: a 2 = 1 Orthogonal vectors: a b = 0 Orthonormal vectors: unit and orthogonal Orthogonal matrix: Orthonormal rows & columns A A = AA = I

36 Skip eigendecomposition, SVD, pseudo-inverse, determinants (Sections ). We will get back to them if/when needed in the course.

37 On to probabilities Chapter 3 of Deep Learning I ve adapted some of the lecture slides from the book. Thanks to Ian Goodfellow for providing slides.

38 Why probabilities? To capture uncertainty E.g., What time will I get home tonight? Probabilities provide a formalism for making statements about data generating processes (L. Wasserman) E.g., what happens when I flip a fair coin? Probability Data generating process Observed data Inference

39 The example Generate data by throwing a fair die. What do we know about a single throw? 6 possible outcomes. (sample space) Each outcome (e.g., 1). (element, state) A subset of outcomes (e.g., <3). (event) Outcomes are equiprobable. (uniform distribution)

40 Random variables and probabilities A random variable (r.v.) is a probabilistic outcome. For example, Die throw (X) The actual outcome is {1, 2, 3, 4, 5, 6}. (x) A probability function (P) assigns a real number to each possible event: P(x) 0, x X P x = 1

41 <latexit sha1_base64="xudvadcuhlkgwfqr9uwkndxf7yo=">aaadfhicfvjnb9naen2yr2k+wjhysygqpviu2qgjlqbktkihckeibaq4isabsbrketfahzdelv8eby7wu7ghrtz5i5xzpwhvcwkktz7em5l9450kk8jsgp5sefeuxrt+y+umf+v2nbv3tnfun1idg449rqu2/qqssqgwr4ik9jodkcyst5pzy0o/pudjhvbvazhhmiwpehpbgrzv77bmz+cjstfaboadcbnbjohwomlw0r3tnh7fy83zfbvxcdyoojcjyqggbjdy+nfumqm+gykohfsqoh0ws8nlsouyctdrxh1fwzk9xffaau0itvxmcnrm17wk/jc2ygnybfgileweil9cnmllqdqopg/gwianuxaaubhoa8dpwaan949qtyrazwgs2wzjynfoc0nc6a+1wyqlgqx5nz3nsna9xjvw0pwmlp5ubjfsekqavthcey7obwrvmme/qutbya0dmrvk26/dc6n2k4m429soqfu7dh+lzjp9ef7x6nk2hx+ge0qdx67kbyao1kaid8sirpoksxfylr7blgh9lzbbyenofhaid0+a+y9wo7pfhrjhrmui9pttsypwzt3gmwqf2wf2xfvkffw+ed8vur3gquybq4x34zdf4aht</latexit> <latexit sha1_base64="xudvadcuhlkgwfqr9uwkndxf7yo=">aaadfhicfvjnb9naen2yr2k+wjhysygqpviu2qgjlqbktkihckeibaq4isabsbrketfahzdelv8eby7wu7ghrtz5i5xzpwhvcwkktz7em5l9450kk8jsgp5sefeuxrt+y+umf+v2nbv3tnfun1idg449rqu2/qqssqgwr4ik9jodkcyst5pzy0o/pudjhvbvazhhmiwpehpbgrzv77bmz+cjstfaboadcbnbjohwomlw0r3tnh7fy83zfbvxcdyoojcjyqggbjdy+nfumqm+gykohfsqoh0ws8nlsouyctdrxh1fwzk9xffaau0itvxmcnrm17wk/jc2ygnybfgileweil9cnmllqdqopg/gwianuxaaubhoa8dpwaan949qtyrazwgs2wzjynfoc0nc6a+1wyqlgqx5nz3nsna9xjvw0pwmlp5ubjfsekqavthcey7obwrvmme/qutbya0dmrvk26/dc6n2k4m429soqfu7dh+lzjp9ef7x6nk2hx+ge0qdx67kbyao1kaid8sirpoksxfylr7blgh9lzbbyenofhaid0+a+y9wo7pfhrjhrmui9pttsypwzt3gmwqf2wf2xfvkffw+ed8vur3gquybq4x34zdf4aht</latexit> <latexit sha1_base64="xudvadcuhlkgwfqr9uwkndxf7yo=">aaadfhicfvjnb9naen2yr2k+wjhysygqpviu2qgjlqbktkihckeibaq4isabsbrketfahzdelv8eby7wu7ghrtz5i5xzpwhvcwkktz7em5l9450kk8jsgp5sefeuxrt+y+umf+v2nbv3tnfun1idg449rqu2/qqssqgwr4ik9jodkcyst5pzy0o/pudjhvbvazhhmiwpehpbgrzv77bmz+cjstfaboadcbnbjohwomlw0r3tnh7fy83zfbvxcdyoojcjyqggbjdy+nfumqm+gykohfsqoh0ws8nlsouyctdrxh1fwzk9xffaau0itvxmcnrm17wk/jc2ygnybfgileweil9cnmllqdqopg/gwianuxaaubhoa8dpwaan949qtyrazwgs2wzjynfoc0nc6a+1wyqlgqx5nz3nsna9xjvw0pwmlp5ubjfsekqavthcey7obwrvmme/qutbya0dmrvk26/dc6n2k4m429soqfu7dh+lzjp9ef7x6nk2hx+ge0qdx67kbyao1kaid8sirpoksxfylr7blgh9lzbbyenofhaid0+a+y9wo7pfhrjhrmui9pttsypwzt3gmwqf2wf2xfvkffw+ed8vur3gquybq4x34zdf4aht</latexit> <latexit sha1_base64="xudvadcuhlkgwfqr9uwkndxf7yo=">aaadfhicfvjnb9naen2yr2k+wjhysygqpviu2qgjlqbktkihckeibaq4isabsbrketfahzdelv8eby7wu7ghrtz5i5xzpwhvcwkktz7em5l9450kk8jsgp5sefeuxrt+y+umf+v2nbv3tnfun1idg449rqu2/qqssqgwr4ik9jodkcyst5pzy0o/pudjhvbvazhhmiwpehpbgrzv77bmz+cjstfaboadcbnbjohwomlw0r3tnh7fy83zfbvxcdyoojcjyqggbjdy+nfumqm+gykohfsqoh0ws8nlsouyctdrxh1fwzk9xffaau0itvxmcnrm17wk/jc2ygnybfgileweil9cnmllqdqopg/gwianuxaaubhoa8dpwaan949qtyrazwgs2wzjynfoc0nc6a+1wyqlgqx5nz3nsna9xjvw0pwmlp5ubjfsekqavthcey7obwrvmme/qutbya0dmrvk26/dc6n2k4m429soqfu7dh+lzjp9ef7x6nk2hx+ge0qdx67kbyao1kaid8sirpoksxfylr7blgh9lzbbyenofhaid0+a+y9wo7pfhrjhrmui9pttsypwzt3gmwqf2wf2xfvkffw+ed8vur3gquybq4x34zdf4aht</latexit> <latexit sha1_base64="xudvadcuhlkgwfqr9uwkndxf7yo=">aaadfhicfvjnb9naen2yr2k+wjhysygqpviu2qgjlqbktkihckeibaq4isabsbrketfahzdelv8eby7wu7ghrtz5i5xzpwhvcwkktz7em5l9450kk8jsgp5sefeuxrt+y+umf+v2nbv3tnfun1idg449rqu2/qqssqgwr4ik9jodkcyst5pzy0o/pudjhvbvazhhmiwpehpbgrzv77bmz+cjstfaboadcbnbjohwomlw0r3tnh7fy83zfbvxcdyoojcjyqggbjdy+nfumqm+gykohfsqoh0ws8nlsouyctdrxh1fwzk9xffaau0itvxmcnrm17wk/jc2ygnybfgileweil9cnmllqdqopg/gwianuxaaubhoa8dpwaan949qtyrazwgs2wzjynfoc0nc6a+1wyqlgqx5nz3nsna9xjvw0pwmlp5ubjfsekqavthcey7obwrvmme/qutbya0dmrvk26/dc6n2k4m429soqfu7dh+lzjp9ef7x6nk2hx+ge0qdx67kbyao1kaid8sirpoksxfylr7blgh9lzbbyenofhaid0+a+y9wo7pfhrjhrmui9pttsypwzt3gmwqf2wf2xfvkffw+ed8vur3gquybq4x34zdf4aht</latexit> <latexit sha1_base64="qqamk0eaqxemjspazau5fqosd54=">aaadf3icfvjnaxrbeo0dv+l4lejry+asilasmylosqimmiprcg4s2flctw/tppme7qg7jmzp5md48ki/xzt49egf8wzpzpxmrlgww+o9qurx1zwvulik45+d4mrva9dvrn0mb92+c/fe+sb9q6srw3hatdtmoaoluigckccjx6vbkdkjr1n+stgpztbyodv7mpu4kmcqxerwie8n04kb7pla5fxjejfux/oivkgyaf22iiotjc6vdkx5vaailshayrkxnhjgshcjdzhwfkvgouxx6kgcau3izt3x0aznxtfeg/8piubs5qohhbwzivozbdcpxdya8l/asklj85etqqwifb84afljihtudcaac4oc5mwd4ez4rxe/bt8g8mnqnzjpnrnktgfgwkxxvjke0r9af3nzayxynntekch1gjdysedkoa43u4tugfdn3d0eyjp0pib6gxx+vx3frt7aevnbtvfav5jqvtki+eovkla/ff+z6d3th+f/rs5nh+ko+qc0uo9l3pbosw1culu7tgmszw63rko/kcnyxqycwyf9jo4n7552t18sdmanpwsp2bzl2do2zfbyarswzjt7yd6zl8gn4gvwlfh+krp0fjupwcuch78bawid+q==</latexit> <latexit sha1_base64="qqamk0eaqxemjspazau5fqosd54=">aaadf3icfvjnaxrbeo0dv+l4lejry+asilasmylosqimmiprcg4s2flctw/tppme7qg7jmzp5md48ki/xzt49egf8wzpzpxmrlgww+o9qurx1zwvulik45+d4mrva9dvrn0mb92+c/fe+sb9q6srw3hatdtmoaoluigckccjx6vbkdkjr1n+stgpztbyodv7mpu4kmcqxerwie8n04kb7pla5fxjejfux/oivkgyaf22iiotjc6vdkx5vaailshayrkxnhjgshcjdzhwfkvgouxx6kgcau3izt3x0aznxtfeg/8piubs5qohhbwzivozbdcpxdya8l/asklj85etqqwifb84afljihtudcaac4oc5mwd4ez4rxe/bt8g8mnqnzjpnrnktgfgwkxxvjke0r9af3nzayxynntekch1gjdysedkoa43u4tugfdn3d0eyjp0pib6gxx+vx3frt7aevnbtvfav5jqvtki+eovkla/ff+z6d3th+f/rs5nh+ko+qc0uo9l3pbosw1culu7tgmszw63rko/kcnyxqycwyf9jo4n7552t18sdmanpwsp2bzl2do2zfbyarswzjt7yd6zl8gn4gvwlfh+krp0fjupwcuch78bawid+q==</latexit> <latexit sha1_base64="qqamk0eaqxemjspazau5fqosd54=">aaadf3icfvjnaxrbeo0dv+l4lejry+asilasmylosqimmiprcg4s2flctw/tppme7qg7jmzp5md48ki/xzt49egf8wzpzpxmrlgww+o9qurx1zwvulik45+d4mrva9dvrn0mb92+c/fe+sb9q6srw3hatdtmoaoluigckccjx6vbkdkjr1n+stgpztbyodv7mpu4kmcqxerwie8n04kb7pla5fxjejfux/oivkgyaf22iiotjc6vdkx5vaailshayrkxnhjgshcjdzhwfkvgouxx6kgcau3izt3x0aznxtfeg/8piubs5qohhbwzivozbdcpxdya8l/asklj85etqqwifb84afljihtudcaac4oc5mwd4ez4rxe/bt8g8mnqnzjpnrnktgfgwkxxvjke0r9af3nzayxynntekch1gjdysedkoa43u4tugfdn3d0eyjp0pib6gxx+vx3frt7aevnbtvfav5jqvtki+eovkla/ff+z6d3th+f/rs5nh+ko+qc0uo9l3pbosw1culu7tgmszw63rko/kcnyxqycwyf9jo4n7552t18sdmanpwsp2bzl2do2zfbyarswzjt7yd6zl8gn4gvwlfh+krp0fjupwcuch78bawid+q==</latexit> <latexit sha1_base64="qqamk0eaqxemjspazau5fqosd54=">aaadf3icfvjnaxrbeo0dv+l4lejry+asilasmylosqimmiprcg4s2flctw/tppme7qg7jmzp5md48ki/xzt49egf8wzpzpxmrlgww+o9qurx1zwvulik45+d4mrva9dvrn0mb92+c/fe+sb9q6srw3hatdtmoaoluigckccjx6vbkdkjr1n+stgpztbyodv7mpu4kmcqxerwie8n04kb7pla5fxjejfux/oivkgyaf22iiotjc6vdkx5vaailshayrkxnhjgshcjdzhwfkvgouxx6kgcau3izt3x0aznxtfeg/8piubs5qohhbwzivozbdcpxdya8l/asklj85etqqwifb84afljihtudcaac4oc5mwd4ez4rxe/bt8g8mnqnzjpnrnktgfgwkxxvjke0r9af3nzayxynntekch1gjdysedkoa43u4tugfdn3d0eyjp0pib6gxx+vx3frt7aevnbtvfav5jqvtki+eovkla/ff+z6d3th+f/rs5nh+ko+qc0uo9l3pbosw1culu7tgmszw63rko/kcnyxqycwyf9jo4n7552t18sdmanpwsp2bzl2do2zfbyarswzjt7yd6zl8gn4gvwlfh+krp0fjupwcuch78bawid+q==</latexit> <latexit sha1_base64="qqamk0eaqxemjspazau5fqosd54=">aaadf3icfvjnaxrbeo0dv+l4lejry+asilasmylosqimmiprcg4s2flctw/tppme7qg7jmzp5md48ki/xzt49egf8wzpzpxmrlgww+o9qurx1zwvulik45+d4mrva9dvrn0mb92+c/fe+sb9q6srw3hatdtmoaoluigckccjx6vbkdkjr1n+stgpztbyodv7mpu4kmcqxerwie8n04kb7pla5fxjejfux/oivkgyaf22iiotjc6vdkx5vaailshayrkxnhjgshcjdzhwfkvgouxx6kgcau3izt3x0aznxtfeg/8piubs5qohhbwzivozbdcpxdya8l/asklj85etqqwifb84afljihtudcaac4oc5mwd4ez4rxe/bt8g8mnqnzjpnrnktgfgwkxxvjke0r9af3nzayxynntekch1gjdysedkoa43u4tugfdn3d0eyjp0pib6gxx+vx3frt7aevnbtvfav5jqvtki+eovkla/ff+z6d3th+f/rs5nh+ko+qc0uo9l3pbosw1culu7tgmszw63rko/kcnyxqycwyf9jo4n7552t18sdmanpwsp2bzl2do2zfbyarswzjt7yd6zl8gn4gvwlfh+krp0fjupwcuch78bawid+q==</latexit> Discrete RVs An RV is discrete if it takes a finite number of values 1 P(x = x i ) 0, i E.g., uniform distribution: i P(x = x i )=1 P(x = x i ) 1 k P(x = x i )= 1 k, i E.g., Poisson distribution: x i exp P(x = x i ; )= x i! 1. technically: it must be countable both images are from: wikipedia.org

42 Continuous RVs An RV is continuous if f(x) 0, x X f(x)dx = 1 b P(a < x < b) = a f(x)dx f(x) is a probability density function (PDF) E.g., (continuous) uniform distribution: u(x = x i ; a, b) = 1 b a E.g., Gaussian distribution from: wikipedia.org

43 A few useful properties (shown for discrete variables for simplicity) Sum rule: P(x) = y P(x, y) Product rule: P(x, y) =P(x y)p(y) Chain rule: P(x 1,...,x n )=P(x 1 )P(x 2 x 1 )P(x 3 x 2, x 1 ) = P(x 1 ) P(x i x 1,...,x i 1 ) If x and y are independent: P(x, y) =P(x)P(y) Bayes Rule: P(Y X) = P(X Y)P(Y) P(X)

44 <latexit sha1_base64="js8ou7gs2gl0dn4g0ziansvjgmw=">aaadq3icfvlnihnbeo6mf2v8y+rrs2nysgaim6lorvjyxd2dqyuy3zvmcd2dstkkp3vorokzhnksn8adf30cn8gbidfbnmyutsiwdppxfvxvvduvp1jydikvne/s5stxr21dr9+4eev2ncb23rorm8ohy7xu5ixmfqrq0ewbes5sayyjjzzg071kp52bsukrt5in0e/ywimr4awdnwg8jsxet+z+3qbpadqyjbcrwhylpt0rw2f+u3zzrfamezay0yxiy0gjgxschdfnec5bkyztelbd+xknnc8sumgls7yxbin2c2zqcallpcospixp2rh6diqwgo0xi/5kuuoyir1p445cumavrhqsstzpyuezmjzyda0i/6x1mhw97rdcprmc4ucpjtjjudnqwhqodhcuuqomg+fqpxzc3jdqjxtllvjrkbly+swylvtjjley/x6lswjrqap8lz1nsna9hdvw4hwnk+s7kqvmmfbv78uhybm4ohh9brn8vv3esm7ti7fa6790v6v8fwzg2t4iwcl35c4lrmb8i/wv1uxverqp7isnhlmq1yk4upsiond7uiwj4+kglotuu8l1vdgejw87ydaj3zxq7j5f7swwuu8ekbyjyroysw7jmekstj6qj+qz+ej98r55370f565ebrlzj6yy9+s3yhau4g==</latexit> <latexit sha1_base64="js8ou7gs2gl0dn4g0ziansvjgmw=">aaadq3icfvlnihnbeo6mf2v8y+rrs2nysgaim6lorvjyxd2dqyuy3zvmcd2dstkkp3vorokzhnksn8adf30cn8gbidfbnmyutsiwdppxfvxvvduvp1jydikvne/s5stxr21dr9+4eev2ncb23rorm8ohy7xu5ixmfqrq0ewbes5sayyjjzzg071kp52bsukrt5in0e/ywimr4awdnwg8jsxet+z+3qbpadqyjbcrwhylpt0rw2f+u3zzrfamezay0yxiy0gjgxschdfnec5bkyztelbd+xknnc8sumgls7yxbin2c2zqcallpcospixp2rh6diqwgo0xi/5kuuoyir1p445cumavrhqsstzpyuezmjzyda0i/6x1mhw97rdcprmc4ucpjtjjudnqwhqodhcuuqomg+fqpxzc3jdqjxtllvjrkbly+swylvtjjley/x6lswjrqap8lz1nsna9hdvw4hwnk+s7kqvmmfbv78uhybm4ohh9brn8vv3esm7ti7fa6790v6v8fwzg2t4iwcl35c4lrmb8i/wv1uxverqp7isnhlmq1yk4upsiond7uiwj4+kglotuu8l1vdgejw87ydaj3zxq7j5f7swwuu8ekbyjyroysw7jmekstj6qj+qz+ej98r55370f565ebrlzj6yy9+s3yhau4g==</latexit> <latexit sha1_base64="js8ou7gs2gl0dn4g0ziansvjgmw=">aaadq3icfvlnihnbeo6mf2v8y+rrs2nysgaim6lorvjyxd2dqyuy3zvmcd2dstkkp3vorokzhnksn8adf30cn8gbidfbnmyutsiwdppxfvxvvduvp1jydikvne/s5stxr21dr9+4eev2ncb23rorm8ohy7xu5ixmfqrq0ewbes5sayyjjzzg071kp52bsukrt5in0e/ywimr4awdnwg8jsxet+z+3qbpadqyjbcrwhylpt0rw2f+u3zzrfamezay0yxiy0gjgxschdfnec5bkyztelbd+xknnc8sumgls7yxbin2c2zqcallpcospixp2rh6diqwgo0xi/5kuuoyir1p445cumavrhqsstzpyuezmjzyda0i/6x1mhw97rdcprmc4ucpjtjjudnqwhqodhcuuqomg+fqpxzc3jdqjxtllvjrkbly+swylvtjjley/x6lswjrqap8lz1nsna9hdvw4hwnk+s7kqvmmfbv78uhybm4ohh9brn8vv3esm7ti7fa6790v6v8fwzg2t4iwcl35c4lrmb8i/wv1uxverqp7isnhlmq1yk4upsiond7uiwj4+kglotuu8l1vdgejw87ydaj3zxq7j5f7swwuu8ekbyjyroysw7jmekstj6qj+qz+ej98r55370f565ebrlzj6yy9+s3yhau4g==</latexit> <latexit sha1_base64="js8ou7gs2gl0dn4g0ziansvjgmw=">aaadq3icfvlnihnbeo6mf2v8y+rrs2nysgaim6lorvjyxd2dqyuy3zvmcd2dstkkp3vorokzhnksn8adf30cn8gbidfbnmyutsiwdppxfvxvvduvp1jydikvne/s5stxr21dr9+4eev2ncb23rorm8ohy7xu5ixmfqrq0ewbes5sayyjjzzg071kp52bsukrt5in0e/ywimr4awdnwg8jsxet+z+3qbpadqyjbcrwhylpt0rw2f+u3zzrfamezay0yxiy0gjgxschdfnec5bkyztelbd+xknnc8sumgls7yxbin2c2zqcallpcospixp2rh6diqwgo0xi/5kuuoyir1p445cumavrhqsstzpyuezmjzyda0i/6x1mhw97rdcprmc4ucpjtjjudnqwhqodhcuuqomg+fqpxzc3jdqjxtllvjrkbly+swylvtjjley/x6lswjrqap8lz1nsna9hdvw4hwnk+s7kqvmmfbv78uhybm4ohh9brn8vv3esm7ti7fa6790v6v8fwzg2t4iwcl35c4lrmb8i/wv1uxverqp7isnhlmq1yk4upsiond7uiwj4+kglotuu8l1vdgejw87ydaj3zxq7j5f7swwuu8ekbyjyroysw7jmekstj6qj+qz+ej98r55370f565ebrlzj6yy9+s3yhau4g==</latexit> <latexit sha1_base64="js8ou7gs2gl0dn4g0ziansvjgmw=">aaadq3icfvlnihnbeo6mf2v8y+rrs2nysgaim6lorvjyxd2dqyuy3zvmcd2dstkkp3vorokzhnksn8adf30cn8gbidfbnmyutsiwdppxfvxvvduvp1jydikvne/s5stxr21dr9+4eev2ncb23rorm8ohy7xu5ixmfqrq0ewbes5sayyjjzzg071kp52bsukrt5in0e/ywimr4awdnwg8jsxet+z+3qbpadqyjbcrwhylpt0rw2f+u3zzrfamezay0yxiy0gjgxschdfnec5bkyztelbd+xknnc8sumgls7yxbin2c2zqcallpcospixp2rh6diqwgo0xi/5kuuoyir1p445cumavrhqsstzpyuezmjzyda0i/6x1mhw97rdcprmc4ucpjtjjudnqwhqodhcuuqomg+fqpxzc3jdqjxtllvjrkbly+swylvtjjley/x6lswjrqap8lz1nsna9hdvw4hwnk+s7kqvmmfbv78uhybm4ohh9brn8vv3esm7ti7fa6790v6v8fwzg2t4iwcl35c4lrmb8i/wv1uxverqp7isnhlmq1yk4upsiond7uiwj4+kglotuu8l1vdgejw87ydaj3zxq7j5f7swwuu8ekbyjyroysw7jmekstj6qj+qz+ej98r55370f565ebrlzj6yy9+s3yhau4g==</latexit> Moments Expectation: E[X] = P(x = x i )x i E[aX] =ae[x] i Variance: 2 = E[(X E[X]) 2 ] Covariance: Cov(X, Y) =E (X E[X])(Y E[Y]) Cov(X, Y) correlation: (x, y) = x y

45 What is the goal of ML?

46 A bit of historical context When I started my PhD very few in ML talked about AI Recent ML makes progress toward AI tasks Examples of AI tasks: translation, object recognition In that context: create a machine with human-like capacities? Or a machine that can help humans?

47 My (imperfect) view: Understand data through predictive models Understand the world through predictive models

48 Further Reading Prologue to The Master Algorithm Ch. 1 of Hastie et al. Math Preparation Ch.2 of Bishop Ch.2-3 of Goodfellow et al. Slightly more advanced:

49 Why did you enroll? Why are you taking this class? What do you hope to learn from it? How do you plan on using it? Any particular topics you would like us to cover?