HARNESSING THE WISDOM OF CROWDS

Size: px

Start display at page:

Download "HARNESSING THE WISDOM OF CROWDS"

Frederica May
5 years ago
Views:

1 1 HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News & Finance Conference March 8, 2017

2 2 Many important decisions in life are made in a group FOMC meeting Board meeting Jury Ask the audience Who wants to be a millionaire

3 Wisdom of crowds Jelly beans in the jar experiment Ask a group of people to guess: 106 135 99 How

80 77 95 jelly beans A large group s average answer to a question involving quantity estimation is

3 3 Wisdom of crowds Jelly beans in the jar experiment Ask a group of people to guess: How many jelly beans in the jar? jelly beans A large group s average answer to a question involving quantity estimation is generally as good as, and often better than, the answer provided by any individual in that group Law of large number requires independence

4 4 Sequential setting: Pros vs. Cons CONS: MEN THINK IN HERDS 95 Herding can reduce the accuracy of a group s average answer The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it s possible that we could become individually smarter but collectively dumber. ---James Surowiecki, The Wisdom of Crowds.

5 5 None of us is as dumb as all of us.

6 6 Sequential setting: Pros vs. Cons CONS: MEN THINK IN HERDS 95 Herding can reduce the accuracy of a group s average answer The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it s possible that we could become individually smarter but collectively dumber. ---James Surowiecki, The Wisdom of Crowds. PROS:GENERATE ADDITIONAL INFORMATION PRODUCTION Additional information production may improve the accuracy of private signals. We become both individually smarter and collectively smarter.

7 7 In this paper WE STUDY THE QUESTION: Do individuals herd in a sequential setting and reduce the usefulness of information aggregated across individuals (i.e., the wisdom of crowds)? WE QUANTIFY THE IMPACT OF HERDING ON ECONOMIC OUTCOMES The empirical challenge is that individual s information set is usually unobservable We overcome the empirical challenge by directly measuring and randomizing on individual s information set

8 8 Our setting A crowd-based earnings forecast platform (Estimize.com) WHY IT S A GOOD SETTING? Forecasts and realizations are clearly measured and easily observable The forecasters do not have direct influence on realizations Corporate earnings are of crucial importance

9 9 Roadmap 1 Herding behavior The influence of herding on forecast accuracy o Individual forecast DATA ANDSTATISTICS o Consensus forecast 3 RANDOMIZED EXPERIMENT Are users herding more with influential users? Does herding lead to return predictability Estimize.com related Sample statistics 2 MAIN ANALYSIS Herding behavior The influence of herding on forecast accuracy o Individual forecast o Consensus forecast 4 INFLUENTIAL USER AND HERDING

10 10

11 11 Estimize.com Open web-based platform founded in 2011, where users can make earnings forecasts Diverse user group: buy-side sell-side independent analysts other working professionals students Estimize consensus is more accurate than WS IBES consensus, also complementary Jame et. al. (2016) Adebambo and Bliss (2015) Now available on Bloomberg Various incentives for making forecasts Competition, monetary and professional prizes Reputation, useful track record Altruism, social preference Scoring system [-25, 25]: - Positive if more accurate than WS consensus - Awarded on an exponential scale, which encourages independent opinion

12 12 Sample statistics Sample period for : Main analysis: 2012/ /03 Randomized experiment: Q2 and Q3 in 2015 Ticker-quarter-forecast panel: 2147 quarterly earnings 2516 users covering 730 stocks: mostly large-growth An average release (a sequence of forecasts for a ticker-quarter): 20 forecasts from 16 users User activity on Estimize.com is tracked by Mixpanel Events: release page, estimate page, submit estimates, etc. Timestamp, location, device, etc.

13 13 Release page view Viewing activity = 1: If a user spent more than 5 seconds on the release page before making her own forecast

14 14 Roadmap 1 Herding behavior The influence of herding on forecast accuracy o Individual forecast DATA ANDSTATISTICS o Consensus forecast 3 RANDOMIZED EXPERIMENT Are users herding more with influential users? Does herding lead to return predictability Estimize.com related Sample statistics 2 MAIN ANALYSIS Herding behavior The influence of herding on forecast accuracy o Individual forecast o Consensus forecast 4 INFLUENTIAL USER AND HERDING

15 15 Herding behavior Individuals herd more when they view others forecasts More weight on the consensus forecast after viewing the release page Information weighting regression, Chen and Jiang (2006, RFS) FE 0 Dev 0 0 : overweight on public information 0 0 : overweight on private information Note: Fixed effects subsume the need to control for stock or user characteristics Clustered standard errors account for autocorrelations in forecast errors

16 16 Influence of herding on forecast accuracy Herding makes individual forecast more accurate but reduces the accuracy of the consensus forecast

17 17 Influence of herding on forecast accuracy Herding makes individual forecast more accurate Individual s absolute forecast error decreases after viewing release page Note: Nonzero Views = 1: if a user spent more than 5 seconds on the release page before making her own forecast CTA dummy = 1: if the forecast is submitted during the last three days before announcements

18 18 Influence of herding on forecast accuracy Herding makes consensus forecast less accurate Consensus s absolute forecast error increases as there are more viewing activities within a release Note: LnNumView = ln (1+ the percentage of forecasts with release views) Std Dev(FE)/Abs(Median(FE)) controls for uncertainty Magnitude: X ln(1+1) = 3.82 cents This is more than the distance between the perfect forecast and the forecast with median Abs(FE) (3 cents) The consensus of group without viewing activity wins more than 50% with statistical significance

19 19 Influence of herding on bias persistence More viewing activity in the close-toannouncement period, the biases in these two periods are more likely to be consistent. Early period Close-to-announcement period Note: Consistent bias indicator = 1: if the bias in both early period and CTA period are in the same direction. LnNumView = ln (1+ the percentage of forecasts with release views)

20 20 Endogeneity concern VIEWING ACTIVITY MAY BE ENDOGENOUS Less informed users are more likely to view others forecasts. Consistent with less accurate consensus. But inconsistent with more accurate individual forecast. ADDRESS THIS CONCERN: randomizing users information sets

21 21 Roadmap 1 Herding behavior The influence of herding on forecast accuracy o Individual forecast DATA ANDSTATISTICS o Consensus forecast 3 RANDOMIZED EXPERIMENT Are users herding more with influential users? Does herding lead to return predictability Estimize.com related Sample statistics 2 MAIN ANALYSIS Herding behavior The influence of herding on forecast accuracy o Individual forecast o Consensus forecast 4 INFLUENTIAL USER AND HERDING

22 22 Blind experiments RANDOMIZED EXPERIMENTS Pilot round (Q2 in 2015): 13 stocks are randomly selected Second round (Q3 in 2015): 90 stocks are randomly selected WHAT WE DO Randomly select users and disable the release page Ask them to make earnings forecasts (blind forecasts) Afterwards, the release page is restored They can immediately revise their forecasts (revised forecasts) Others not selected still view the original release page and make forecasts (default forecasts)

23 23 Blind view

24 24 Blind vs. Default User characteristics Users in blind and default group are similar in observable user characteristics. Note: Estimate view, Avg #releases, #tickers, and abs(fe) are based on users forecasts before the experiment

25 25 Blind vs. Default Herding and consensus accuracy HERDING BEHAVIOR Default forecasts put more weight on Estimize consensus relative to blind forecasts Placebo test with concurrent WS consensus CONSENSUS ACCURACY (HORSE RACE) Blind consensus significantly beats default consensus 56.3% of the time Default consensus significantly beats blind consensus 36.9% of the time

26 26 Blind vs. Revised Consensus forecast accuracy The blind consensus is much more accurate than the revised consensus Note: This test only includes releases in the pilot round, because only in the pilot round, most users revise their forecasts immediately after the release page is restored. Information weighting regression

27 27 Roadmap 1 Herding behavior The influence of herding on forecast accuracy o Individual forecast DATA ANDSTATISTICS o Consensus forecast 3 RANDOMIZED EXPERIMENT Are users herding more with influential users? Does herding lead to return predictability Estimize.com related Sample statistics 2 MAIN ANALYSIS Herding behavior The influence of herding on forecast accuracy o Individual forecast o Consensus forecast 4 INFLUENTIAL USER AND HERDING

users in general Viewed by another influential user D views A s estimate

28 28 Measuring user influence Page rank algorithm based on viewing activities B and C are influential users in this network Viewed by many users in general Viewed by another influential user D views A s estimate Factors behind user influence: Note: page rank is also how Google ranks websites.

29 29 Influential user and herding Information weighting regression Individuals put more weight on the consensus if it contains the forecasts of influential users Note: Influenced = 1 if the number of influential users ahead of the observed user is above the 80th percentile across all observations Robustness check using data from the experiment

30 30 Predicting forecast errors of influential users SENTIMENT-BASED MEASURE Influential users revision: upward downward Contemporaneous firm returns: positive negative

31 31 Predicting earnings announcement returns Market does not seem to undo the optimism bias completely and is negatively surprised on average at the earnings announcement

32 Conclusions Herding improves the accuracy of individual forecast but reduces the accuracy of consensus forecast Wisdom of crowds can be better

32 32 Conclusions Herding improves the accuracy of individual forecast but reduces the accuracy of consensus forecast Wisdom of crowds can be better harnessed by encouraging independent voice Aftermath: Our experiment results convinced Estimize.com to switch to a blind platform in November 2015

Harnessing the Wisdom of Crowds

Harnessing the Wisdom of Crowds Zhi Da, and Xing Huang First Draft: May 2015 This Draft: September 2015 Abstract By tracking user viewing activities on a crowd-based earnings forecast platform (Estimize.com),