A Little History of Machine Learning

Transcription

1 機器學習現在 過去 未來 A Little History of Machine Learning Chia-Ping Chen National Sun Yat-sen December 2016

2 Outline ubiquitous machine intelligence challenge and reaction AI brief deep learning conclusion

3 ubiquitous machine intelligence

4 San Francisco: 673 MUNI

5 Seoul: Traveling Salesman

6 Just Very Recently advertisement map immigration bus/train/thsr i-card bridge information system vs. intelligent system big data vs. machine learning

7 The Largest Slice of Pie device (hardware + software) networking machine learning

8 Who Wants to Do Machine Learning web giants device manufacturers internet service providers social networks e-commerce chip design/manufacture company Google + Facebook + Amazon + IBM + MS

9 challenge and reaction

10 Human vs. Machine Machine beats non-experts, even experts. speaking painting singing playing musical instruments playing games driving cars question answering special effects

11 New Work Forces Provide (replace) manpowers in stock market medicine law education middle-class jobs (usa 47% china 77%)

12 Human in Machine Era acknowledge the difference between human and machine use machines know machine learning deep better than broad creative better than routine do what you like and like what you do humanity heart emotion

13 AI brief

14 Legends Church-Turing thesis A function on the natural numbers is computable by a human being following an algorithm, ignoring resource limitations, if and only if it is computable by a Turing machine. Turing test A human being should be unable to distinguish the machine from another human being by using the replies to questions put to both.

15 Data Mining

16 Pattern Recognition Human PR Brahe Kepler Newton (observation) (description) (explanation) Machine PR computer computer who cares? (transactions) (association) (hindsight)

17 Machine Learning Tasks and Performances anomaly detection classification regression transcription translation synthesis parsing imputation denoising density estimation

18 deep learning

19 AI Milestones Deep Blue (not deep learning) Deep QA (still not deep learning) AlphaGo by DeepMind (deep learning)

20 What Is Deep Learning Neural networks with many layers of units. Very successful VoiceSearch (automatic speech recognition) WaveNet (speech synthesis) ImageNet (image classification) Translator (machine translation) AlphaGo (honorary 10 dan) LeCun, Bengio, and Hinton, Deep Learning, Nature, 2015 D. Silver, Aja Huang, et al. Mastering the game of Go with deep neural networks and tree search, Nature, 2016

21 Future of Machine Learning robots (Tokyo 2020) imitating human creativity (synthesis of art works) unsupervised learning zero-data learning long-term and long-range problems (programming, writing) going after really difficult problems (weather, earthquake, astronomy, brain, cancer, medicine)

22 a research work at NSYSU MIT Lab

23 Background accepted for oral presentation at ICASSP 2016 joint work with Hon Hai Technology Group simple idea, good results sleepless in San Franscisco

24 Problem Statement task: hand-written digit recognition data: images of hand-written digits with added noises (a) (b) (c) (d) (e) (f) goal: noise-robustness model: artificial neural network

25 Artificial Neural Network input layer for the pixels x = {x i } hidden layer to bridge between input and output h = {h k } output layer for the digits y = {y j }

26 Propagation of Information linear combination of input pixel values M N a k = w k,mn x mn m=1 n=1 non-linear activation function h k = σ(a k ) linear combination of hidden unit values K a j = w jk h k k=1 transform a j to output value y j

27 Vector vs. Image An image can be represented by a vector, and vice versa. image to vector x x 1N..... x M1... x MN x = x 1. x I vector to image w k,11. w k,mn = w k w k,11... w k,1n..... w k,m1... w k,mn

28 Visualization Visualize a hidden unit k by incoming link weights w k,mn. 30 images, each corresponds to a hidden unit. left block: without orthogonalization. right block: with orthogonalization.

29 Inner Product and Projection The linear combination I a k = w ki x i i=1 can be seen as an inner product a k = w T k x or as a projection, noting that the projection of x to v is v T x v T v v

30 Orthogonality We often want to work with an orthogonal basis. orthogonality makes projection simple orthogonality makes projection economic orthogonality makes projection independent

31 Gram-Schmidt Process Gram-Schmidt process converts a set of linearly independent vectors v 1,..., v J to a set of orthonormal vectors by orthogonalization w 1,..., w J j 1 u j = v j v j, u i u i i=1 and normalization w j = u j u j

32 System Framework Recognition Model Recognition Model Output layer (10 neurons) Output layer (10 neurons) Hidden layer (30 neurons) Input layer (784 neurons) Image (28 28 pixels) Hidden layer (30 neurons) Gram-Schmidt process Input layer (784 neurons) Image (28 28 pixels) baseline proposed

33 MNIST images of hand-written digits, each with pixels I = classes for digit 0 to digit 9 J = training examples and test examples

34 Examples random difficult

35 Noisy Data (a) (b) (c) (d) (e) (f) clean and noisy data with 0-20 db SNR

36 Weight Images At a Glance

37 Results baseline SNR accuracy imp. over clean clean 95.1 = 20 db db db db db proposed SNR accuracy imp. over baseline 20 db db db db db

38 Visualization of Hidden Layer h 1,..., h 6, h i = σ(a i ) h 1,..., h 6, h i = σ(a i ) Hamming distance between clean and noisy data baseline system 2, 1, 1, 0, 0 proposed system 1, 1, 0, 0, 0

39 Summary machine learning today brief AI history booming deep learning a recent research work at NSYSU current machine learning NSYSU MIT Lab EmoDB and FAU-Aibo MNIST Aurora Twitters Board games