Modeling Controversy Within Populations

Transcription

1 Modeling Controversy Within Populations Myungha Jang, Shiri Dori-Hacohen and James Allan Center for Intelligent Information Retrieval (CIIR) University of Massachusetts Amherst {mhjang, shiri, ICTIR 2017, Amsterdam, Netherlands

2 Social Media and Controversy Social media are increasingly the place where controversial arguments are being held. Technological tools have become critical in shaping these discussions. However, serious gaps remain in our theoretical and practical understanding of how to define controversy, and how it manifests and evolves. 2 / 20 mhjang@cs.umass.edu

3 Definition of Controversy What makes controversy? Is the topic important? Is there enough contention? 3 / 20 mhjang@cs.umass.edu

4 Related Work Quantifying controversy in Social Media using a community graph structure [Garimella et al. WSDM 16] Detecting controversy in Web pages using the level of related Wikipedia articles edit war [Dori-Hacohen et al. ECIR 13] Detecting controversy in Web pages using language model [Jang et al. CIKM 16] Detecting Controversies in Online News Media using a debase-based approach [Beelan et al. SIGIR 17] 4 / 20

5 Controversy as Single Absolute Value? Quantifying controversy in Social Media using a community graph structure [Garimella et al. WSDM 16] While exploiting different features, previous works share the same underlying assumption that controversy is a single absolute value the Detecting for topic. controversy in Web pages using the level of related Wikipedia articles edit war [Dori-Hacohen et al. ECIR 13] Detecting controversy in Web pages using language model [Jang et al. CIKM 16] Detecting Controversies in Online News Media using a debase-based approach [Beelan et al. SIGIR 17] 4 / 20

6 Definition of Controversy Is controversy really an absolute, single value for amorphous global population? A disparity between scientific understanding and public opinion on certain controversial issues A general population Scientists 5 / 20 mhjang@cs.umass.edu

7 Definition of Controversy Is controversy really an absolute, single value for amorphous global population? A disparity between scientific understanding and public opinion on certain controversial issues Should we reduce the page limit of full paper to 8 pages at SIGIR? A general population IR Community 5 / 20 mhjang@cs.umass.edu

8 Definition of Controversy Is controversy really an absolute, single value for amorphous global population? A disparity between scientific understanding and public opinion on certain controversial issues Should we reduce the page limit of full paper to 8 pages at SIGIR? A general population IR Community 5 / 20 mhjang@cs.umass.edu

9 Our Contributions We propose a theoretical model that defines controversy in terms of population. The right question to be asked is not Is climate change controversial? but is climate change controversial to {a particular group}? We model controversy as multidimensional with at least two dimensions: Contention Importance Some topics may be less controversial because they are contentious but not important (or vice versa). Our model has explanatory power that can be used to understand a variety of observed phenomena. To ground our theoretical model, we examine a diverse dataset from polling data, Twitter and Wikipedia. 6 / 20 mhjang@cs.umass.edu

10 Modeling Population-based Controversy We model P Controversy T, Ω) where T is a topic and Ω is a population. P Controversy T, Ω) = P Contention, Importance T, Ω) Assuming that contention and importance are independent to each other: P Controversy T, Ω) = P Contention T, Ω P(Important T, Ω) 7 / 20 mhjang@cs.umass.edu

11 Modeling Contention (1/5) Contention is a measure that quantifies the proportion of people in disagreement within a population. Ω = {p V, p W p Y } is a population consisting of N people P c Ω, T is a probability of T is contentious within Ω and is modeled by: P Controversy T, Ω) = P Contention T, Ω P(Important T, Ω) 8/ 20 mhjang@cs.umass.edu

12 Modeling Contention (1/4) Contention is a measure that quantifies the proportion of people in disagreement within a population. Ω = {p V, p W p Y } is a population consisting of N people P c Ω, T is a probability of T is contentious within Ω and is modeled by: P(c Ω, T) = P(p 1, p 2 selected randomly from Ω, s i, s j S, s. t. holds p 1, s i, T holds p 2, s j, T P conflicts s i, s j Where S = {s 1, s 2 } is a set of stances with regard to T holds(p, s, T): person p holds stance s with regard to T s 0 is a lack of stance: I don t have a stance Let P(conflict s i, s j ) be the probability that two groups of s j and s k are in a complete conflict. P Controversy T, Ω) = P Contention T, Ω P(Important T, 9 / 20 Ω) 9/ 20 mhjang@cs.umass.edu

13 Modeling Contention (2/4) P(c Ω, T) = P(p 1, p 2 selected randomly from Ω, s i, s j S, s. t. holds p 1, s i, T holds p 2, s j, T P conflicts s i, s j Mutually Exclusive Stances Most controversial topics have at least two mutually exclusive stances to bisect the community. Ω = ~ G j j {.. } Where G i is a group that consists of people who hold s j For simplicity, we estimate the probability of selecting p V and p W as selection with replacement. The probability of choosing any particular pair is V }. P Controversy T, Ω) = P Contention T, Ω P(Important T, 10 / 20 Ω) 10 / 20 mhjang@cs.umass.edu

14 Modeling Contention (3/4) Mutually Exclusive Stances One person doesn t have a stance and the other has a stance Two people have the same stance Two people have conflicting stances P Controversy T, Ω) = P Contention T, Ω P(Important T, 11 / 20 Ω) 11/ 20 mhjang@cs.umass.edu

15 Modeling Contention (3/4) Mutually Exclusive Stances One person doesn t have a stance and the other has a stance Two people have the same stance Two people have conflicting stances Finally, P Controversy T, Ω) = P Contention T, Ω P(Important T, 11 / 20 Ω) 11/ 20 mhjang@cs.umass.edu

16 Modeling Contention (4/4) Mutually Exclusive Stances Contention G1 Contention is maximized when the all groups are in the same size. The bigger G is, the smaller the maximal contention becomes in the given population. P Controversy T, Ω) = P Contention T, Ω P(Important T, 12 / 20 Ω) 12/ 20 mhjang@cs.umass.edu

17 Modeling Importance Importance is the level of impact that the issue brings to the world within the population. How many people are affected by the topic? Let affected(p, T) be a binary function that denotes whether person p is affected by T. P I Ω, T = P(p selected randomly from Ω affected (p, T)) = Ω T Ω P Controversy T, Ω) = P Contention T, Ω P(Important T, Ω) 13/ / 20 mhjang@cs.umass.edu

18 Modeling Importance Importance is the level of impact that the issue brings to the world within the population. How many people are affected by the topic? Let affected(p, T) be a binary function that denotes whether person p is affected by T. P I Ω, T = P(p selected randomly from Ω affected (p, T)) = Ω T The subpopulation that is affected by T Ω The function is purposely defined vaguely such that it can be instantiated differently depending on the dataset. In Twitter, affected user posts a tweet about the event. In Wikipedia, affected editor makes changes to the article. P Controversy T, Ω) = P Contention T, Ω P(Important T, Ω) 14/ / 20 mhjang@cs.umass.edu

19 Model Validation: Contention in Polling US Adults vs Scientists Pew Research Center Gallup Data in 2015 Contention in the scientific community vs. the general population for several controversial topics. 14 / 20 mhjang@cs.umass.edu

20 Model Validation: Contention in Polling US Adults vs Scientists Issues whose contention levels align well between the scientists and the U.S. adults group Opinion on the increased use of fracking {favor / against} The space station has been {good / bad investment} for the country: Do you think it is generally {safe / unsafe} to eat foods grown with pesticides? Contention in the scientific community vs. the general population for several controversial topics. 14 / 20 mhjang@cs.umass.edu

21 Model Validation: Contention in Polling US Adults vs Scientists Issues whose contention is in disparity between the scientists and th U.S. adults group Do you support using animals in Research? {support, against} What is the reason for climate change? {Earth is warming mostly due to human activity vs Earth is warming mostly due to natural patterns} Has evolution really existed? {Yes, No} Contention in the scientific community vs. the general population for several controversial topics. 14 / 20 mhjang@cs.umass.edu

22 Model Validation: Contention in Polling US Adults vs Scientists Issues whose contention is in disparity between the scientists and th U.S. adults group Most scientists believe that using animals in research is necessary, whereas it is more controversial between U.S. adults. Most scientists believe that climate change is caused by human activity whereas it is controversial between U.S. adults with more people believing that it is a natural pattern. Most scientists believe that evolution is real whereas some people don t believe it. Contention in the scientific community vs. the general population for several controversial topics. 14 / 20 mhjang@cs.umass.edu

23 Model Validation: Contention in Polling Population by State Answers collected by isidewith.com, an on-line opinion gathering site. Population of each state provides different Ω for each. The green states of Wyoming, Alabama, Mississippi, etc are the states that are strongly opposed to gun control. Per-state contention for Do you support increased gun control? More demo: 15 / 20 mhjang@cs.umass.edu

24 Model Validation Controversy In Twitter Given a topic T, we measure controversy in Twitter population. We assume that the user u is affected by T if u posts anything with a topic hashtag. G1: Supports Hillary G2: Supports Trump G0: a lack of stance (or neutral). 16 / 20 mhjang@cs.umass.edu

25 Model Validation Controversy In Twitter Given a topic T, we measure controversy in Twitter population. We assume that the user u is affected by T if u posts anything with a topic hashtag. G1: Supports Hillary G2: Supports Trump G0: a lack of stance (or neutral) We first identify the subpopulation in which T is important to them, and classify people into stance groups (G1, G2 ) and G0. Note that here, G0 is not a group for a lack of stance to T, but a group that is affected by T but does not hold a particular stance. The details such as (1) finding relevant hashtags for T, (2) identifying the sub-population of interest, and (3) finding stance groups, are in the paper. 16 / 20 mhjang@cs.umass.edu

26 Model Validation Controversy In Twitter 17 / 20 mhjang@cs.umass.edu

27 Model Validation Controversy In Twitter Buzzfeed article 17 / 20 mhjang@cs.umass.edu

28 Model Validation Controversy In Twitter Buzzfeed article Voting day 17 / 20 mhjang@cs.umass.edu

29 Model Validation Controversy In Twitter 17 / 20 mhjang@cs.umass.edu

30 Model Validation Controversy In Twitter According to our model, U.S. Election was far more controversial than Brexit and the Dress on Twitter. 18 / 20 mhjang@cs.umass.edu

31 Model Validation Controversy In Twitter According to our model, The Dress was more contentious for one day but Brexit was more important. 18 / 20 mhjang@cs.umass.edu

32 Conclusions and Future Work In contrast to past work, we propose: a theoretical model for controversy with respect to population. a model that models controversy as multi-dimensional with two minimal dimensions, contention and importance. We validate our model using the poll and Twitter data by showing our model can explain the observed phenomena. Future work includes: Using the model to detect newly emerged controversy, not just validating it Adding more dimensions of controversy to the model 19 / 20 mhjang@cs.umass.edu

33 Thank you! And thanks to ACM SIGIR for the travel grant! Our demo and dataset for download: 20 / 20 mhjang@cs.umass.edu