Bayesian Social Learning with Random Decision Making in Sequential Systems

Transcription

1 Bayesian Social Learning with Random Decision Making in Sequential Systems Yunlong Wang supervised by Petar M. Djurić Department of Electrical and Computer Engineering Stony Brook University Stony Brook, NY 11794, USA September 15, 2014 Yunlong Wang Bayesian Social Learning with Random Decision Making 1/37

2 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 2/37

3 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 3/37

4 Interest We consider a large number of agents that get observations from an hypothesis that can be either H 0 and H 1. In a predefined order, they make decisions sequentially between the two hypotheses. The decisions of agents are made according to their beliefs on the true state of nature. For each agent, its decision can be observed by all the agents. The agents learn from the previous agents decisions and formulate their beliefs by using the Bayesian theory. Yunlong Wang Bayesian Social Learning with Random Decision Making 4/37

5 Interest Decision making in the proposed sequential system Yunlong Wang Bayesian Social Learning with Random Decision Making 5/37

6 Interest We want to find weather herd behavior and information cascade can take place in this system. We want to show that asymptotic learning can be achieved, i.e., that the probability an agent makes right decision converges to one. Yunlong Wang Bayesian Social Learning with Random Decision Making 6/37

7 Motivation An example of herd behavior Yunlong Wang Bayesian Social Learning with Random Decision Making 7/37

8 Our argument A large body of literature investigates this problem with the purpose of maximizing some utility function, where the agents are assumed to be selfish. Then once the belief of one selfish agent has been formed, the decision will be made deterministically. However, in some scenarios, the behaviors of agents may be by design random. For example, there is a chance that an agent refuses to behave as predicted because of some random disturbance such as mood, misleading information, etc. Our contribution lies in that we analyze the properties of such systems when the agents behave in a random manner. Yunlong Wang Bayesian Social Learning with Random Decision Making 8/37

9 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 9/37

10 The setup A network of N Bayesian agents make decisions between two hypotheses H 0 and H 1 in a predefined order. Each agent has its private observation generated either by H 1 or H 0. Every agent s decision can be observed by all the following agents. The agents form their beliefs on the state of nature by using Bayes rule. Yunlong Wang Bayesian Social Learning with Random Decision Making 10/37

11 The network model The sequential system can be depicted by the following diagram: Yunlong Wang Bayesian Social Learning with Random Decision Making 11/37

12 The observation model Each agent A n receives its private observation y n that is generated according to one of the following two hypotheses: H 1 : y n p(y n H 1 ) H 0 : y n p(y n H 0 ). Let w n (k) = p(y n H k ), k {0, 1} be the data likelihood for each hypothesis, then the log(w n (1) /w n (0) ) represents the log likelihood ratio (LLR) of data y n. The boundedness of the LLR will play an important role in social learning: a bounded LLR can be overwhelmed by a strong prior. Yunlong Wang Bayesian Social Learning with Random Decision Making 12/37

13 Let α n {0, 1} be the decision of agent A n and α 1:n be the decision sequence from A 1 to A n. Then agent A n can form its private belief β n = p(h 1 α 1:n 1, y n ) by using the Bayes rule: β n = π n 1 w n (1) π n 1 w n (1) + (1 π n 1 )w n (0) where π n 1 = p(h k α 1:n 1 ) denotes the social belief. The social belief π n 1 serves as the prior knowledge of agent A n before it has its private observation y n. For agent A 1, the initial social belief is defined to be π 0 = 1/2. Yunlong Wang Bayesian Social Learning with Random Decision Making 13/37

14 The decision policy Agent A n makes its decision from a certain distribution conditioned on its private belief β n, i.e. α n p(α n β n ). For example, agents can make decision 1 whenever β n > 0.5 and 0 otherwise. In this work, we address a specific random decision making policy. We propose that after agent A n obtaining its private belief, it makes its decision according to, { β n, if α n = 1, p(α n β n ) = 1 β n, O.W., where the decision α n of A n is a Bernoulli random variable parameterized by the A n s private belief β n. Yunlong Wang Bayesian Social Learning with Random Decision Making 14/37

15 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 15/37

16 Bayesian learning Learning from actions: Social belief update In this system, once agent A n makes its decision α n, all the following agents should update the social belief by Bayes rule from π n 1 = p(h 1 α 1:n 1 ) to p(h 1 α 1:n ). The social belief is updated according to π n = ( where k {0, 1}, π n 1 (1 l n (1) ) π n 1 (1 l n (1) ) + (1 π n 1 )(1 l (0) ( denotes the action likelihood. π n 1 l n (1) π n 1 l n (1) + (1 π n 1 )l n (0) l (k) n = p(α n = 1 α 1:n 1, H k ) n ) ) αn, ) 1 αn Yunlong Wang Bayesian Social Learning with Random Decision Making 16/37

17 Bayesian learning The action likelihood By marginalizing the private observation y n, it can be shown that, l (k) n = p(α n = 1 α 1:n 1, y n )p(y n H k )dy n. We point out that the above sequential learning algorithm belongs to the social learning filter. Yunlong Wang Bayesian Social Learning with Random Decision Making 17/37

18 Information cascade and herds Definitions We start the analysis on the proposed system with the two definitions: An agent A n herds on the public belief π n 1 if it makes its decision α n independently of its observation y n, i.e., p(α n = 1 α 1:n 1, y n ) = p(α n = 1 α 1:n 1 ). An information cascade occurs at time ñ if the public belief stops evolving after agent Añ, i.e., π n = πñ, n > ñ. Yunlong Wang Bayesian Social Learning with Random Decision Making 18/37

19 Information cascade and herds The action likelihood and herd behavior If agent A n herds on the public belief π n 1, then the action likelihood will be independent of the true state of nature, i.e., l n (0) = l n (1), where l n (k) = p(α n = 1 α 1:n 1, H k ). If l (0) n = l (1) n, it holds that π n = π n 1. If every agent after agent Añ herds on the public belief, there is an information cascade starting with the agent Añ. If l n (0) l n (1), the agent A n must not herd on the public belief, and its private information is partially conveyed by its decision α n. Yunlong Wang Bayesian Social Learning with Random Decision Making 19/37

20 Information cascade and herds The action likelihood and herd behavior(cont.) When the agents make their decisions according to the proposed model, the action likelihood can be written as l n (k) π n 1 w n (1) = π n 1 w n (1) + (1 π n 1 )w n (0) w n (k) dy n, by which we have the following lemma: Lemma With the proposed random policy, if the two distributions w n (1) and w n (0) are not identical everywhere and if 1 > π n > 0, n N +, we have that l n (1) l n (0) 0, n N +. The above lemma shows that no agent herds on the public belief. Yunlong Wang Bayesian Social Learning with Random Decision Making 20/37

21 Asymptotical properties Asymptotical learning In sum, we have shown that the social belief never stops evolving. We next show that the social belief will converge to the true state of nature by the following theorem: Theorem In the proposed sequential random decision making system, the expected value of social belief asymptotically converge to 1 when the true state of nature is H s = H 1, and 0 otherwise, i.e., lim Eπ n = n { 1, if H s = H 1, 0, if H s = H 0. Yunlong Wang Bayesian Social Learning with Random Decision Making 21/37

22 Asymptotical properties Asymptotical learning(cont.) Considering the symmetric structure of the proposed system, we assume that the true state of nature is H 1. Then it can be shown that Eπ n is increasing in terms of n, which is given by Eπ n Eπ n 1 = p(α 1:n 1 H 1 ) (α 1:n 1 ), α 1:n 1 A n 1 where (α 1:n 1 ) = π n 1 Z (1 π n 1) 2 (l (1) n l (0) n ) 2 > 0. By the above equations, the limit of Eπ n exists. Noting that l n (1) l n (0), the Eπ n Eπ n 1 is strictly greater than zero unless Eπ n = 1. Yunlong Wang Bayesian Social Learning with Random Decision Making 22/37

23 Asymptotical properties Asymptotical learning(cont.) Theorem Suppose that the agents implement the random decision policy. Let H s be the true state of nature, then s {0, 1}, we have lim p(α n = s) = 1. n The probability that the agents make the right decision converges to one even if the LLR is bounded. Yunlong Wang Bayesian Social Learning with Random Decision Making 23/37

24 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 24/37

25 Introduction In this part we want to further analyze the proposed random system by comparison with other two systems. First we describe the three systems, and then we explain some of their features. Due to lack of time, we do not show the detailed derivations on the evolution of beliefs in those systems. Yunlong Wang Bayesian Social Learning with Random Decision Making 25/37

26 Deterministic policy In cases where the agents implement the deterministic policy, for agent A n, its decision is made according to { 1, if β n > 0.5, α n = 0, O.W., By using this deterministic policy, the agents myopically optimize their personal utility. Yunlong Wang Bayesian Social Learning with Random Decision Making 26/37

27 Deterministic policy(cont.) When the LLR is bounded, the following statements hold: The space of social belief S = [0, 1] can be divided into three non empty subsets S 0,S r, and S 1. Once the social belief evolves into S 0 or S 1, all the following agents will make decision as zero or one with probability one, and information cascade will start. Social belief evolves to S 0 or S 1 in finite time with probability one. On the other hand, if the LLR is unbounded, we have that the error probability will converge to zero. Yunlong Wang Bayesian Social Learning with Random Decision Making 27/37

28 Random policy Compared with the system that uses deterministic policy, in a random system even if the LLR is bounded, the region S 0 and S 1 do not exist. This property prevents the information cascade. As was shown by Lemma 1, no agent ignores its private observation no matter what the social belief it uses as its prior in forming its private belief. Yunlong Wang Bayesian Social Learning with Random Decision Making 28/37

29 Hybrid policy We also propose that the agent A n can make its decision by a hybrid policy, where 1, if β n > c, α n = R n, if c > β n > 1 c, 0, O.W., with R n Ber(β n ) being a Bernoulli random variable parameterized by β n, and with 1 > c > 0.5 denoting a threshold for the agent to make a decision either randomly or deterministically. The deterministic and random policies are two special cases of the hybrid policy when c = 0.5 and c = 1, respectively. The asymptotical property of this policy is identical with that of the deterministic policy. Yunlong Wang Bayesian Social Learning with Random Decision Making 29/37

30 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 30/37

31 Experiment 1: Binomial model Consider the data model given by H 1 : y n B(m, p 1 ), H 0 : y n B(m, p 0 ), where B(m, p) denotes a binomial distribution parameterized by m and p. It can be shown that the LLR of this model is bounded. In the first experiment, we verified the analytical result of the expected social belief by implementing the three policies with the binomial data for 2,000 trials. In each trial, the parameters were set to be N = 1, 500, m = 5, p 0 = 0.6, and p 1 = 0.7. When using the hybrid policy, we set c = Yunlong Wang Bayesian Social Learning with Random Decision Making 31/37

32 Result 1: Evolution of social belief with the binomial model Average social belief from all the 2,000 trials plotted as a function of agent number for three policies Eπ n deterministric hybrid random agent number Yunlong Wang Bayesian Social Learning with Random Decision Making 32/37

33 Experiment 2: Gaussian model Consider a system where the agents get observations from the Guassian data model: H 1 : y n N (θ 1, σ 2 w ), H 0 : y n N (θ 0, σ 2 w ), where y n is a Gaussian random variable parameterized by θ 0 = 0 or θ 1 = θ. The parameters θ and σ w are known by the agents. The LLR of this Gaussian data model is unbounded. Same as in the first experiment, in experiment 2, we implemented the three policies with the Gaussian data for 2000 trials. The parameters here were set to be N = 1, 500, σ w = 2, θ = 1 and c = Yunlong Wang Bayesian Social Learning with Random Decision Making 33/37

34 Result 2: Evolution of social belief with Gaussian model The convergence of expected social belief can be achieved by all three policies Eπ n deterministric hybrid random agent number Yunlong Wang Bayesian Social Learning with Random Decision Making 34/37

35 Experiment 3: Outliers Evolution of social belief with the sequence of decisions where α 1:40 all equal to one except that α 10 = π n deterministic method, α 10 =0 hybrid method, α 10 =0 random method, α 10 = agent number Yunlong Wang Bayesian Social Learning with Random Decision Making 35/37

36 Outline Introduction Problem formulation Analysis Comparisons with other systems Simulation results Conclusions Yunlong Wang Bayesian Social Learning with Random Decision Making 36/37

37 Concluding remarks We presented a random model for the social learning system. The agents implement the Bayesian method to learn from previous agents actions and the private signal. After analyzing the herd behavior and information cascade, we showed the convergence of social belief. We demonstrated the performance of the proposed method by applying it to the binomial and Gaussian data models. We used Monte Carlo simulations to verify the analytical results. Yunlong Wang Bayesian Social Learning with Random Decision Making 37/37