Scott E Page. University of Michigan Santa Fe Institute. Leveraging Diversity

Transcription

1 Scott E Page University of Michigan Santa Fe Institute Leveraging Diversity

2 Outline Big Challenges Kiddie Pools Cattle and Netflix Problem Solving

3 Poverty Climate Change Energy Health Care Disease/Epidemics Global Finance Education

4 Claim We need diverse thinkers to meet these challenges

5 Support Make logical, scientific arguments

6

7 E[SqE(c)] = B n n 1 Var + n Cov

8

9 Kiddie Pools

10

11 Experiment Four Types of Plants: A,B,C,D

12 Three types of Pools Homogeneous: AAA, BBB, CCC, DDD Mixed: AAB, ABB, AAC, ACC, AAD, ADD, BBC, BCC, BBD, BDD, CCD, CDD Diverse: ABC, ABD, ACD, BCD

13 Suppose A s Make Pool Robust Homogeneous: AAA, BBB, CCC, DDD (25%) Mixed: AAB, ABB, AAC, ACC, AAD, ADD, BBC, BCC, BBD, BDD, CCD, CDD (50%) Diverse: ABC, ABD, ACD, BCD (75%)

14 Suppose A and B Make Pool Robust Homogeneous: AAA, BBB, CCC, DDD (0%) Mixed: AAB, ABB, AAC, ACC, AAD, ADD, BBC, BCC, BBD, BDD, CCD, CDD (17%) Diverse: ABC, ABD, ACD, BCD (50%)

15 Inescapable Benefits of Diversity THEOREM: If the marginal contribution to some outcome: robustness, efficiency, innovation, etc diminishes in the number of each type (the second of a type contributes less than the first A and so on) then

16 Inescapable Benefits of Diversity THEOREM: If the marginal contribution to some outcome: robustness, efficiency, innovation, etc diminishes in the number of each type (the second of a type contributes less than the first A and so on) then on average diverse collections will do best See Page Diversity and Complexity 2010

17

18

19 Prediction

20 X lb?

21 The Spherical Cow

22 The Gateway Cow

23 Diversity Prediction Theorem Crowd Error = Average Error - Diversity

24 Crowd Error = Average Error Diversity 0.6 = 2,

25 Case: Netflix Prize

26 Netflix Prize $1M to anyone who can defeat their Cinematch recommender system by 10% of more.

27 Details Netflix users rank using stars Six years of data Half million users 17,700 movies Data divided into (training, testing) Testing Data dived into (probe, quiz, test)

28 Interesting Asides Lost in Translation, The Royal Tenenbaums had the highest variance Shawshank Redemption had the highest rating Miss Congeniality had the most ratings.

29 BellKor s Initial Models Approximately 50 dimensions Best Model: 6.8% improvement Combination of Models: 8.4% improvement

30 BellKor s Pragmatic Chaos More is Better: Seven person team Functional Diversity: statisticians, machine learning experts and computer engineers Identity Diversity: United States, Austria, Canada and Israel. Difficult be build a grand model (800 variables) but possible to build lots of huge models

31 Ensemble Effects Best Model 8.4% Ensemble: 10.1% Rules: Once someone breaks 10%, then the contest ends in 30 days.

32 Enter ``The Ensemble 23 teams from 30 countries who blended their predictive models who tried in the last moments to defeat BellKor s Pragamatic Chaos

33 And The Winner is RMSE The Ensemble: Bellkor's Pragmatic Chaos

34 Oh, by the way.. BellKor s Pragmatic Chaos 10.06% The Ensemble 10.06% 50/50 Blend 10.19%

35 Problem Solving

36 First 15 Nobels in Physics: 19 Winners 1901 Wilhelm C. Röntgen 1902 Hendrik A. Lorentz, Pieter Zeeman 1903 Antoine Henri Becquerel, Pierre Curie, Marie Curie 1904 John W. Strutt 1905 Philipp E. A. von Lenard 1906 Sir Joseph J. Thomson 1907 Albert A. Michelson 1908 Gabriel Lippmann 1909 Carl F. Braun, Guglielmo Marconi 1910 Johannes D. van der Waals 1911 Wilhelm Wien 1912 Nils G. Dalen 1913 Heike Kamerlingh Onnes 1914 Max von Laue 1915 William Bragg

37 42

38 Foldit

39 Polymath Project

40 Density Hales-Jewett theorem. How many boxes must you remove to make tic-tac-toe impossible? X X X

41 Density Hales-Jewett theorem. How many boxes must you remove to make tic-tac-toe impossible? X X X

42 A test. Create a bunch of agents with diverse perspectives and heuristics Rank them by their performance on a problem. Note: all of the agents must be smart

43 Group 1 Best 20 agents Group 2 Random 20 agents Have each group work collectively. When one agent gets stuck at a point, another agent tries to find a further improvement. Group stops when no one can find a better solution.

44 The IQ View Alpha Group Diverse Group

45 The diverse group almost always outperforms the group of the best by a substantial margin. See Lu Hong and Scott Page Proceedings of the National Academy of Sciences (2002)

46 The Toolbox View Alpha Group Diverse Group ABC ACD BDE AHK FD AEG BCD ADE BCD EZ BCD IL

47 What Must be True? Calculus Condition Problem solvers must all be smart we must be able to list their local optima Diversity Condition Problem solvers must have diverse heuristics and perspectives Hard Problem Condition Problem itself must be difficult

48

49

50 Final Thought

51 Value of Diversity Depends on Extent of Collaboration and Nature of the Problem

52

53 Boosting

54 Talent + Difference