Rating Protocols in Online Communities

Transcription

1 Rating Protool in Online Communitie Yu Zhang, Jaeo Par, Mihaela van der Shaar 3 Abtrat Sutaining ooperation among elf-intereted agent i ritial for the proliferation of emerging online ommunitie. Providing inentive for ooperation in online ommunitie i partiularly hallenging beaue of their unique feature: a large population of anonymou agent having aymmetri interet and dynamially joining and leaving the ommunity; operation error; and agent trying to whitewah when they have a low tanding in the ommunity. In thi paper, we tae thee feature into onideration and propoe a framewor for deigning and analyzing a la of inentive heme baed on rating protool, whih onit of a rating heme and a reommended trategy. We firt define the onept of utainable rating protool under whih every agent ha inentive to follow the reommended trategy given the deployed rating heme. We then formulate the problem of deigning an optimal rating protool, whih elet the protool that maximize the overall oial welfare among all utainable rating protool. Uing the propoed framewor, we tudy the truture of optimal rating protool and explore the impat of one-ided rating, punihment length and whitewahing on optimal rating protool. Our reult how that optimal rating protool are apable of utaining ooperation, with the amount of ooperation varying depending on the ommunity harateriti. Categorie and Subjet Deriptor: H.. [Information Sytem Appliation]: General; J. [Soial and Behavioral Siene]: Eonomi General Term: Deign, Eonomi, Human Fator Additional Keyword and Phrae: Inentive heme, online ommunitie, rating heme, reommended trategy, whitewahing. I. INTRODUCTION Reent development in tehnology have expanded the boundarie of ommunitie in whih individual interat with eah other. For example, nowaday individual an obtain valuable information or ontent from remotely loated individual in an online ommunity formed through online networing ervie []-[7]. However, a large population and the anonymity of individual in uh an online ommunity mae it diffiult to utain ooperative behavior among elf-intereted individual [8][9]. For example, it ha been reported that free-riding i widely oberved in peer-to-peer networ [][]. Hene, inentive heme are needed to ultivate ooperative behavior in online ommunitie. A variety of inentive heme have been explored to indue ooperation in uh online ommunitie. The mot popular inentive are baed on priing heme and differential ervie proviion. Priing heme ue payment to reward and punih individual for their behavior, whih in priniple an indue elf-intereted individual to ooperate with eah other to attain the oial optimum by internalizing their external effet (ee, for example, [][3]). However, priing heme often require omplex aounting infratruture, whih introdue ubtantial ommuniation and omputation overhead []. Hene, it i impratial for priing heme and the orreponding infratruture to be implemented in open, ditributed ommunitie, e.g. peer-to-peer networ [7], mobile networ [], et., where agent have limited omputing and ommuniation apabilitie. Moreover, the operator of online ommunitie may be relutant to adopt a priing heme when priing diourage individual partiipation in ommunity ativitie. Finally and mot importantly, the ervie being exhanged in online ommunitie are often not real good but rather olution to mall ta or proviion of mall amount of reoure Jaeo Par i with Shool of Eonomi, Yonei Univerity, Korea. jaeo.par@yonei.a.r Thi wor wa done when Jaeo Par wa a potdotoral holar at Eletrial Engineering Department, Univerity of California, o Angele, USA. 3 Yu Zhang and Mihaela van der Shaar are with Eletrial Engineering Department, Univerity of California, o Angele, USA. yuzhang@ula.edu, mihaela@ee.ula.edu.

2 whih are diffiult to prie. For example, in the online quetion and anwer forum uh a Yahoo! Anwer [], the exhanged ervie repreent a mall favor in anwering the quetion poted by other uer. The diffiulty in priing uh mall ervie in thee appliation prevent priing heme to be effetive. Differential ervie heme, on the other hand, reward and punih individual by providing differential ervie depending on their behavior intead of uing monetary reward [5]-[3]. Differential ervie an be provided by ommunity operator or by ommunity member. Community operator an treat individual differentially (for example, by varying the quality or ope of ervie) baed on the information about the behavior of individual. Inentive proviion by a entral entity an offer a robut method to utain ooperation [5]. However, uh an approah i impratial in a large ommunity beaue the burden of a entral entity to monitor individual behavior and provide differential ervie for them beome prohibitively heavy a the population ize grow. Alternatively, more ditributed inentive heme exit where ommunity member monitor the behavior of eah other and provide differential ervie baed on their obervation [6]-[3]. Suh inentive heme are baed on the priniple of reiproity and an be laified into peronal reiproation (or diret reiproity) [6]-[8] and oial reiproation (or indiret reiproity) [9]-[3]. In peronal reiproation heme, individual an identify eah other, and behavior toward an individual i baed on their peronal experiene with that individual. Peronal reiproation i effetive in utaining ooperation in a mall ommunity where individual an identify eah other and interat frequently with fixed opponent, but it loe it power in a large ommunity where individual have aymmetri interet and an freely and frequently hange the opponent they interat with [6]. In oial reiproation heme, individual obtain ome information about other individual (for example, rating) and deide their ation toward an individual baed on thi available information. Hene, an individual an be rewarded or punihed by other individual in the online ommunity who have not had pat interation with him [6][7]. Therefore, oial reiproation ha a potential to form a bai of ueful inentive heme for online ommunitie. A uh, thi paper i devoted to the tudy of inentive heme baed on oial reiproation. Sutaining ooperation uing oial reiproation ha been invetigated in the literature uing the framewor of anonymou random mathing game, in whih eah individual i repeatedly mathed with different partner over time for ervie exhange and trie to maximize hi diounted long-term utility. To implement oial reiproation, it i important for the ommunity to hare enough information about pat interation uh that the ommunity member now how to reward or punih other. Thi exiting literature mae different aumption on the information revealed to ommunity member about other member. In [8] eah ommunity member oberve the entire hitory of the pat play of hi urrent partner. In [9][3], ommunity member are informed about the outome of the mathe in whih they have been diretly involved. Rating protool have been propoed in [6] and [7], where eah ommunity member i attahed a rating ore indiating hi oial tatu, whih tae a value from a finite et and reord hi pat play, and ommunity member with different rating ore are treated differently by other individual they interat with. For online ommunitie, maintaining diret reord of individual pat play whih are ued in [8]-[3] are not appropriate, beaue the ommuniation and torage ot for revealing the entire hitory of the pat play of an individual grow unbounded with time. Sine the ue of rating ore a a ummary reord require ignifiantly le amount of information to be maintained, we will deign

3 inentive heme baed on rating protool for online ommunitie. A rating-baed inentive heme an be eaily implemented in online ommunitie that deploy entitie (e.g., a traer in PP networ [7], or a web portal in web-baed appliation []) that an ollet, proe, and deliver information about individual play hitory to generate rating ore. Cooperation among ommunity member an be utained in all the above wor on anonymou random mathing game. However, all of them have foued on obtaining the Fol Theorem by haraterizing the et of equilibrium payoff that an be ahieved when the diount fator of individual i uffiiently loe to. Our wor, on the ontrary, addree the problem of deigning a rating-baed inentive heme given a diount fator and other parameter ariing from pratial onideration, whih are not fully onidered in the exiting literature on anonymou random mathing game. Speifially, our wor tae into aount the following feature of online ommunitie: Aymmetry of interet. A an example, onider a ommunity where individual with different area of expertie hare nowledge with eah other. It will be rarely the ae that a pair of individual ha a mutual interet in the expertie of eah other imultaneouly. We allow the poibility of aymmetri interet by modeling the interation between a pair of individual a a gift-giving game, intead of a prioner dilemma game whih aume mutual interet between a pair of individual [7][6][7][8]. Report error. In an inentive heme baed on a rating protool, it i poible that the rating ore (or label) of a peifi individual i updated inorretly beaue of error in the report of hi partner (i.e. other individual he interat with). Our model inorporate the poibility of report error, whih allow u to analyze it impat on deign and performane, wherea mot exiting wor on rating heme (e.g. [6][7]) adopt an idealized aumption that rating ore are alway updated orretly. Dynami hange in the population. The member of a ommunity hange over time a individual gain or loe interet in the ervie provided by ommunity member. We model thi feature by having a fration of individual leave and join the ommunity in every period. Thi allow u to tudy the impat of population turnover on deign and performane. Whitewahing. Whitewahing refer to the behavior of an individual reating multiple identitie by repeatedly entering to an anonymou online ommunity. In an online ommunity, individual with bad rating ore may attempt to whitewah their rating ore by leaving and rejoining the ommunity a new member to avoid the punihment impoed by the ytem upon their old identitie [7]. We onider thi poibility and tudy the deign of whitewah-proof rating protool and their performane. Note that our model and analyi alo differ ignifiantly from mot exiting wor on reputation ytem [3]- [5]. Firt, the model in [3]-[5] aume that individual aume fixed role in the ommunity (i.e. eller or buyer), whih i ommon in appliation where the group of eller and buyer are eparated and uually do not overlap []. Neverthele, in online ommunitie uh a PP networ, online labor maret, et., eah agent an be both the provider and the reeiver of ervie. Seond, the reputation ytem in [3]-[5] rely on differential In thi wor, we peifially ue the aymmetry of interet to refer to the fat that two individual in one tage game do not have mutual interet in the reoure poeed by eah other imultaneouly. The invetigation on other dimenion of individual aymmetry (e.g. heterogeneou individual harateriti) erve a an intereting extenion of thi wor. 3

4 priing heme to inentivize eller to ooperate. We have already mentioned that the ervie being exhanged and hared in online ommunitie are diffiult to prie, thereby preventing uh priing-baed reputation ytem to be effetively deployed. Finally and mot importantly, [3]-[5] onider the repeated game between a unique long-lived eller and many hort-lived buyer, and the deign priniple there i to maximize the expeted (diounted) long-term utility of the individual long-lived eller when the diount fator of the eller and the payment-to-ot ratio are uffiiently large. In ontrat, we onider in thi paper the interplay among a large number of long-lived individual, and aim to maximize the oial welfare (i.e. the um utility) of the entire ommunity for arbitrary diount fator and payment-to-ot ratio, whih mae the deign in [3]-[5] inappliable here. A more detailed and in-depth omparion between our wor and [3]-[5] i provided in Setion VII. The differene between our wor and the exiting literature on oial reiproity are ummarized in the following table in order to highlight our ontribution and novelty. TABE. Comparion between the exiting literature and our wor. [8]-[3][35] [6][7] [3]-[5] Our wor Inentive devie Differential ervie Differential ervie Monetary reward Differential ervie Aymmetry of interet N/A No No Ye Report error N/A No Ye Ye Information requirement Entire hitory of tage game outome Individual rating Individual rating Individual rating Diount fator Suffiiently loe to Suffiiently loe to Suffiiently large Arbitrary Number of longlived player Multiple Multiple One Multiple Protool deign No No Ye Ye Optimization riterion Individual long-term utility Individual long-term utility Individual long-term utility Sum utility of all player The remainder of thi paper i organized a follow. In Setion II, we deribe the repeated anonymou mathing game and inentive heme baed on a rating protool. In Setion III, we formulate the problem of deigning an optimal rating protool. In Setion IV, we provide analytial reult about optimal rating protool. In Setion V, we extend our model to addre the impat of variable punihment length, whitewahing poibility, and one-ided rating. We provide imulation reult in Setion VI, and onlude the paper in Setion VII. II. MODE A. Repeated Mathing Game We onider a ommunity where eah member, or agent, an offer a valuable ervie to other agent. Example of ervie are expert nowledge, utomer review, job information, multimedia file, torage pae, and omputing power. We onider an infinite-horizon direte time model with a ontinuum of agent [7]. 5 In a 5 The ontinuum population model i ommonly adopted in the analyi for large-ale dynami networ, e.g. peer-to-peer ytem [3][7], grid networ [], oial haring webite [][5], et. Alo, it ha been hown in our tehnial report [] that the ontinuum model an ignifiantly redue the omplexity of deigning optimal rating protool in the anonymou random mathing game while inurring mall effiieny lo ompared to the ae where a finite population model i employed.

5 period, eah agent generate a ervie requet [37], whih i ent to another agent that an provide the requeted ervie. 6 We model the requet generation and agent eletion proe uing uniform random mathing: eah agent reeive exatly one requet in every period and eah agent i equally liely to reeive the requet of an agent, and the mathing i independent aro period. 7 Suh model well approximate the mathing proe between agent in large-ale online ommunitie. For example, in a mobile relay networ [] where agent (e.g. mobile devie) within a ertain area are able to relay traffi for eah other through unliened petrum (e.g. WAN) to the detination (e.g. nearby ellular bae tation), the relay node that eah mobile agent enounter at eah moment ould be approximately aumed to be random, ine thi mobile agent i moving around the area randomly over time. In a pair of mathed agent, the agent that requet a ervie i alled a lient while the agent that reeive a ervie requet i alled a erver. In every period, eah agent in the ommunity i involved in two mathe, one a a lient and the other a a erver. Note that the agent with whom an agent interat a a lient an be different from that with whom he interat a a erver, refleting aymmetri interet between a pair of agent in a given intant. We model the interation between a pair of mathed agent a a gift-giving game [3]. In a gift-giving game, the erver ha the binary hoie of whether to fulfill or deline the requet, while the lient ha no hoie. The erver ation determine the payoff of both agent. If the erver fulfill the lient requet, the lient reeive a ervie benefit of b > while the erver uffer a ervie ot of >. We aume that b > o that the ervie of an agent reate a poitive net oial benefit. If the erver deline the requet, both agent reeive zero payoff. The et of ation for the erver i denoted by = { FD, }, where F tand for fulfill and D for deline. The payoff matrix of the gift-giving game i preented in Table. An agent play the gift-giving game repeatedly with hanging partner until he leave the ommunity. We aume that at the end of eah period a fration a Î [,] of agent in the urrent population leave and the ame amount of new agent join the ommunity. We refer to a a the turnover rate [7]. Soial welfare in a time period i meaured by the average payoff of the agent in that period. Sine b >, oial welfare i maximized when all the erver hooe ation F in the gift-giving game they play, whih yield payoff b- to every agent. On the ontrary, ation D i the dominant trategy for the erver in the gift-giving game, whih ontitute a Nah equilibrium of the gift-giving game. When every erver hooe hi ation to maximize hi urrent payoff myopially, an ineffiient outome arie where every agent reeive zero payoff. TABE. Payoff matrix of a gift-giving game. Server F D Client b, -, B. Inentive Sheme Baed on a Rating Protool In order to improve the effiieny of the myopi equilibrium, we ue inentive heme baed on rating 6 It hould be noted that our analyi an be readily extended to the ae where eah agent generate a ervie requet with a probability l <. We aume l = in thi paper only for the impliity of illutration. 7 Uniform random mathing erve a a good aumption and i ommonly adopted in the analyi for online ommunitie with large population where agent provide idential ervie, e.g. wirele relay networ [], peer-to-peer networ [3][6]. The impat of mathing heme on the inentive of agent and the performane of online ommunitie fall out the ope of thi paper, but erve a an important next tep in thi line of reearh. 5

6 protool. A rating protool i defined a the rule that a ommunity ue to regulate the behavior of it member. Thee rule indiate the etablihed and approved way of operating (e.g., exhanging ervie) in the ommunity: adherene to thee rule i poitively rewarded, while failure to follow thee rule reult in (poibly evere) punihment [36]. Thi give rating protool a potential to provide inentive for ooperation. We onider a rating protool that onit of a rating heme and a reommended trategy, a in [6] and [7]. A rating heme determine the rating of agent depending on their pat ation a a erver, while a reommended trategy preribe the ation that erver hould tae depending on the rating of the mathed agent. Formally, a rating heme i repreented by three parameter (, K, t) : denote the et of rating ore that an agent an hold, K Î denote the initial rating ore attahed to newly joining agent, and t i the rating update rule. After a erver tae an ation, the lient end a report (or feedba) about the ation of the erver to the third-party devie or infratruture that manage the rating ore of agent, but the report i ubjet to error with a mall probability e. That i, with probability e, D i reported when the erver tae ation F, and vie vera. Auming a binary et of report, it i without lo of generality to retrit e in [,/ ]. When e = /, report are ompletely random and do not ontain any meaningful information about the ation of erver. We onider a rating heme that update the rating ore of a erver baed only on the rating ore of mathed agent and the reported ation of the erver. Then, a rating heme an be repreented by a mapping t :, where tqq (,, a R ) i the new rating ore for a erver with urrent rating ore q when he i mathed with a lient with rating ore q and hi ation i reported a a R. A reommended trategy i repreented by a mapping :, where qq (, ) i the approved ation for a erver with rating ore q that i mathed with a lient with rating ore q. 8 To implify our analyi, we initially impoe the following retrition on rating heme. 9 ) i a nonempty finite et, i.e., = {,, ¼, } for ome nonnegative integer. ) K =. 3) t i defined by ìï min{ q +, } if a = ( q, q ), R tqq (,, a ) = ï R í ï if a qq (, ). R ¹ ïî () Note that with the above three retrition a nonnegative integer ompletely deribe a rating heme, and thu a rating protool an be repreented by a pair = (, ). We all the rating heme determined by the maximal punihment rating heme (MPRS) with punihment length. In the MPRS with punihment length, there are + rating ore, and the initial rating ore i peified a. If the reported ation of the erver i the ame a that peified by the reommended trategy, the erver rating ore i inreaed by while not exeeding. Otherwie, the erver rating ore i et a. A hemati repreentation of an MPRS i provided 8 The trategie in the exiting rating mehanim [6][7] determine the erver ation baed olely on the lient rating ore, and thu an be onidered a a peial ae of the reommended trategie propoed in thi paper. 9 We will relax the eond and third retrition in Setion V. 6

7 in Fig.. Below we ummarize the equene of event in a time period: ) Agent generate ervie requet and are mathed. ) Eah erver oberve the rating of hi lient and then determine hi ation. 3) Eah lient report the ation of hi erver. ) The rating ore of agent are updated, and eah agent oberve hi new rating ore for the next period. 5) A fration of agent leave the ommunity, and the ame amount of new agent join the ommunity. III. PROBEM FORMUATION A. Stationary Ditribution of Rating Sore A time pae, the rating ore of agent are updated and agent leave and join the ommunity. Thu, the t ditribution of rating ore in the ommunity evolve over time. et h () q be the fration of q -agent in the total population at the beginning of an arbitrary period t, where a q -agent mean an agent with rating q. Suppoe that all the agent in the ommunity follow a given reommended trategy. Then the tranition from { h ( q )} = to t+ { h ( q)} q= i determined by the rating heme, taing into aount the turnover rate a and the error probability e, a hown in the following expreion: t q h t + t+ () = ( -a) e, h () q = (-a)(-e) h ( q- ) for q -, t+ t t h ( ) = (-a)(- e){ h ( ) + h ( - )} + a. t () Sine we are intereted in the long-term payoff of the agent, we tudy the ditribution of rating ore in the long run. Definition (Stationary ditribution) { hq ()} i a tationary ditribution of rating ore under the dynami defined by () if it atifie å hq () =, hq () ³, " q, and q= h() = ( -a) e, hq () = (-a)(-ehq - ) for q -, h( ) = (-a)(- e){ h( ) + h( - )} + a. (3) The following lemma how the exitene of and onvergene to a unique tationary ditribution. emma. For any e Î [,/ ] and a Î [,], there exit a unique tationary ditribution { hq ()} whoe expreion i given by q+ q hq () = (-a) (- e) e, for q -, ìï if a = e =, h( ) = ï + í ( -a) ( - e) e+ a otherwie. ï ïî -(-a)(-e) () Moreover, the tationary ditribution { hq ()} i reahed within ( + ) period tarting from any initial ditribution. 7

8 Proof: Suppoe that a > or e >. Then (3) ha a unique olution whih atifie q+ hq () = (-a) (- e) e, for q -, + ( -a) ( - e) e+ a h( ) =, -(-a)(-e) å hq () =. Suppoe that a = and e =. Then olving (3) together with q= q å q= (5) hq () = yield a unique olution ( ) h q = for q - and h ( ) =. It i eay to ee from the expreion in () that hq () i reahed within ( q + ) period, for all q, tarting from any initial ditribution. Sine the oeffiient in the equation that define a tationary ditribution are independent of the reommended trategy that the agent follow, the tationary ditribution i alo independent of the reommended trategy, a an be een in (). Thu, we will write the tationary ditribution a { h ( q )} to emphaize it dependene on the rating heme, whih i repreented by. B. Sutainable Rating Protool We now invetigate the inentive of agent to follow a preribed reommended trategy. For impliity, we he the inentive of agent at the tationary ditribution of rating ore, a in [7] and [3]. Sine we onider a non-ooperative enario, we need to he whether an agent an improve hi long-term payoff by a unilateral deviation. Note that any unilateral deviation from an individual agent would not affet the evolution of rating ore and thu the tationary ditribution, beaue we onider a ontinuum of agent. et (, qq ) be the ot uffered by a erver with rating ore q that i mathed with a lient with rating ore q and follow a reommended trategy, i.e., (, ) qq = if qq (, ) = F and (, qq ) = if qq (, ) = D. Similarly, let b (, qq ) be the benefit reeived by a lient with rating ore q that i mathed with a erver with rating ore q following a reommended trategy, i.e., b (, ) b qq = if qq (, ) = F and b (, qq ) = if qq (, ) = D. Sine we onider uniform random mathing, the expeted period payoff of a q -agent under rating protool before he i mathed i given by å å v () q = h () q b (, q q) - h () q (, q q). (6) q Î q Î To evaluate the long-term payoff of an agent, we ue the diounted um riterion in whih the long-term payoff of an agent i given by the expeted value of the um of diounted period payoff from the urrent period. et p ( q q) be the tranition probability that a q -agent beome a q -agent in the next period under rating protool. Sine we onider MPRS, p ( q q) an be expreed a Thi i true for any deviation by agent of meaure zero. 8

9 ìï - e if q = min{ q +, }, p ( q q) = ïe if q, for all q í = Î. (7) ï otherwie, ïî Then we an ompute the long-term payoff of an agent from the urrent period (before he i mathed) by olving the following reurive equation v () q = v () q + d p ( q q) v ( q ) å q Î for all q Î, (8) where d = b( - a) i the weight that an agent put on hi future payoff. Sine an agent leave the ommunity with probability a at the end of the urrent period, the expeted future payoff of a q -agent i given by ( a ) p ( q q - å ) v ( q ), auming that an agent reeive zero payoff one he leave the ommunity. The q Î expeted future payoff i multiplied by a ommon diount fator b Î [,), whih reflet the time preferene, or patiene, of agent. Now uppoe that an agent deviate and ue a trategy under rating protool. Sine the deviation of a ingle agent doe not affet the tationary ditribution, the expeted period payoff of a deviating q -agent i given by å v () () (, ) () (, ), q = h q b q q + h q q q. (9) q Î et p ( q q, q ), be the tranition probability that a q -agent uing the trategy beome a q -agent in the next period under rating protool, when he i mathed with a lient with rating ore q. For eah q, q = min{ q +, } with probability ( - e) and q = with probability e if qq (, ) = (, qq ) while the probabilitie are revered otherwie. Then p ( ) ( ) p (, ), q q = h q å, q q q å q Î qî give the tranition probability of a q -agent before nowing the rating ore of hi lient, and the long-term payoff of a deviating agent from the urrent period (before he i mathed) an be omputed by olving v () q = v () q + då p ( q q) v ( q ),,,, q Î for all q Î. () In our model, a erver deide whether to provide a ervie or not after he i mathed with a lient and oberve the rating ore of the lient. Hene, we he the inentive for a erver to follow a reommended trategy at the point when he now the rating ore of the lient. Suppoe that a erver with rating ore q i mathed with a lient with rating ore q. When the erver follow the reommended trategy preribed by rating protool, he reeive the long-term payoff (, ) p ( ) v ( ) qq d q q - + å q, exluding the poible benefit a a lient in q the urrent period. On the ontrary, when the erver deviate to a reommended trategy, he reeive the longterm payoff (, ), (, ) - qq p v + d q qq å, ( q ), again exluding the poible benefit a a lient. By q 9

10 omparing thee two payoff, we an he whether a q -agent ha an inentive to deviate to when he i mathed with a lient with rating ore q. Definition (Sutainable rating protool) A rating protool i utainable if for all, for all ( qq, ). å - (, qq ) + d p ( q q ) v ( q ) ³- (, qq ) + d p ( q qq, ) v ( q ) () q q,, In word, a rating protool = (, ) i utainable if no agent an gain from a unilateral deviation regardle of the rating ore of the lient he i mathed with when every other agent follow reommended trategy and the rating ore are determined by the MPRS with punihment length. Thu, under a utainable rating protool, agent follow the preribed reommended trategy in their elf-interet. Cheing whether a rating protool i utainable uing the above definition require omputing deviation gain from all poible reommended trategie, whoe omputation omplexity an be quite high for moderate value of. By employing the riterion of unimprovability in Marov deiion theory [38], we etablih the one-hot deviation priniple for utainable rating protool, whih provide impler ondition. For notation, let a be the ot uffered by a erver that tae ation a, and let p ( q q, q ) be the tranition probability that a q -agent beome a q -agent in the next period under, a rating protool when he tae ation a to a lient with rating ore q. The value of p ( q q, q ) an be, obtained in a imilar way to obtain p ( q q, q ),, by omparing a with qq (, ). emma (One-hot Deviation Priniple). A rating protool i utainable if and only if (, ) é a { ( ), a (, )} qq d q q q qq ( q ) ù - - êå ú ë q û () for all a ¹ qq (, ), for all ( qq, ). Proof: If rating protool i utainable, then learly there are no profitable one-hot deviation. We an prove the onvere by howing that, if i not utainable, there i at leat one profitable one-hot deviation. Sine (, qq ) and a are bounded, thi i true by the unimprovability property in Marov deiion theory [33][3]. emma how that if an agent annot gain by unilaterally deviating from only in the urrent period and following afterward, he annot gain by withing to any other reommended trategy either, and vie vera. The left-hand ide of () an be interpreted a the urrent gain from hooing a, while the right-hand ide of () repreent the diounted expeted future lo due to the different tranition probabilitie indued by hooing a. Uing the one-hot deviation priniple, we an derive inentive ontraint that haraterize utainable rating protool. Firt, onider a pair of rating ore ( qq, ) uh that qq (, ) = F. If the erver with rating ore q erve the lient, he uffer the ervie ot of in the urrent period while hi rating ore in the next period beome min{ q +, } with probability ( - e) and with probability e. Thu, the expeted long-term payoff of a q - å a

11 agent when he provide a ervie i given by q V ( F; F) =- + d[( - e) v (min{ q +, }) + ev ()] (3) On the ontrary, if a q -agent deviate and deline the ervie requet, he avoid the ot of in the urrent period while hi rating ore in the next period beome with probability ( - e) and min{ q +, } with probability e. Thu, the expeted long-term payoff of a q -agent when he doe not provide a ervie i given by q V ( D; F) = d[( - e) v () + ev (min{ q +, })]. () The inentive ontraint that a q -agent doe not gain from a one-hot deviation i given by V ( F; F) ³ V ( D; F), whih an be expreed a d( - e)[ v (min{ q +, }) - v ()] ³. (5) Now, onider a pair of rating ore ( qq, ) uh that qq (, ) = D. Uing a imilar argument a above, we an how that the inentive ontraint that a q -agent doe not gain from a one-hot deviation an be expreed a d( - e)[ v (min{ q +, }) - v ()] ³-. (6) Note that (5) implie (6), and thu for q uh that qq (, ) = F for ome q, we an he only the firt inentive ontraint (5). Therefore, a rating protool i utainable if and only if (5) hold for all q uh that qq (, ) = F for ome q and (6) hold for all q uh that qq (, ) = D for all q. The left-hand ide of the inentive ontraint (5) and (6) an be interpreted a the lo from punihment that rating protool applie to a q -agent for not following the reommended trategy. Therefore, in order to indue a q -agent to provide a ervie to ome lient, the left-hand ide hould be at leat a large a the ervie ot, whih an be interpreted a the deviation gain. We ue min { d( - e)[ v (min{ q +, }) -v ()]} to meaure the trength of the inentive qî for ooperation under rating protool, where ooperation mean providing the requeted ervie in our ontext. C. Rating Protool Deign Problem Sine we aume that the ommunity operate at the tationary ditribution of rating ore, oial welfare under rating protool an be omputed by U = å h () q v () q. (7) q The ommunity operator aim to hooe a rating protool that maximize oial welfare among utainable rating protool. Then the problem of deigning a rating protool an be formally expreed a maximize U = h ( q ) v ( q ) (, ) å q ubjet to d( - e)[ v (min{ q +, }) - v ()] ³, " quh that $ q uh that ( q, q ) = F, d( - e)[ v (min{ q +, }) - v ()] ³-, " q uh that ( q, q ) = D " q. A rating protool that olve the deign problem (8) i alled an optimal rating protool. q q (8) IV. ANAYSIS OF OPTIMA RATING PROTOCOS

12 A. Optimal Value of the Deign Problem We firt invetigate whether there exit a utainable rating protool, i.e., whether the deign problem (8) ha a feaible olution. Fix the punihment length and onider a reommended trategy D defined by D (, q q ) = D for all ( qq, ). Sine there i no ervie provided in the ommunity when all the agent follow we have v (, ) () q = for all q. Hene, the relevant inentive ontraint (6) i atified for all q, and the rating D D protool (, ) i utainable. Thi how that the deign problem (8) alway ha a feaible olution. Auming that an optimal rating protool exit, let D, U be the optimal value of the deign problem (8). In the following propoition, we tudy the propertie of U. Propoition. The optimal value of the deign problem (8) atifie the following propertie: (i) - e U b-. - e (ii) U = if b b( -a)( - e) >. - b( - a)( - 3 e) (iii) U ³ [ -( -ae ) ]( b - ) if b( -a)( - e). b (iv) (v) U = b- if e = and b( - a). b U = b- only if e = and b b( - a). - b( - a) Proof: See Appendix A. Propoition (i) prove that the optimal oial welfare annot be negative but i alway tritly bounded away from b-, whih i the oial welfare when all agent ooperate, when e >. Hene full ooperation annot be ahieved in thi enario. Sine we obtain zero oial welfare at myopi equilibrium, without uing a rating protool, we are intereted in whether we an utain a rating protool in whih agent ooperate in a poitive proportion of mathe. In other word, we loo for ondition on the parameter ( bbae,,,, ) that yield U >. From Propoition (ii) and (iii), we an regard / b [ b( -a)( - e)]/ [ -b( -a)( - 3 e)] and / b b( -a)(- e) a neeary and uffiient ondition for U >, repetively. Moreover, when there are no report error (i.e., e = ), we an interpret / b b( -a)/[-b(- a)] and / b b( - a) a neeary and uffiient ondition to ahieve the maximum oial welfare U = b-, repetively. A a orollary of Propoition, we obtain the following reult in the limit. Corollary. For any ( b, ) uh that b >, (i) U onverge to b- a b, a, and e, and (ii) U onverge to a b, a, or e /.

13 Corollary how that we an deign a utainable rating protool that ahieve near effiieny (i.e., U loe to b- ) when the ommunity ondition are good (i.e., b i loe to, and a and e are loe to ). Moreover, it uffie to ue only two rating (i.e., = ) for the deign of uh a rating protool. On the ontrary, no ooperation an be utained (i.e., U = ) when the ommunity ondition are bad (i.e., b i loe to, a i loe to, or e i loe to /), a implied by Propoition (ii). B. Optimal Reommended trategie Given a Punihment ength In order to obtain analytial reult, we onider the deign problem (8) with a fixed punihment length, denoted DP. Note that DP ha a feaible olution, namely D, for any and that there are a finite number (total ( ) + ) of poible reommended trategie given. Therefore, DP ha an optimal olution for any. We ue U and to denote the optimal value and the optimal reommended trategy of DP, repetively. We firt how that inreaing the punihment length annot dereae the optimal value. Propoition. U ³ U for all and uh that ³. Proof: See Appendix B. Propoition how that U i non-dereaing in. Sine U < b- when e >, we have U = lim U = up U. It may be the ae that the inentive ontraint eventually prevent the optimal value from inreaing with o that the upremum i attained by ome finite. Thi onjeture i verified in Fig., where U top inreaing when ³ 5. Hene, it i plauible for the protool deigner to et an upper bound on in pratial deign with little effiieny lo inurred. Now we analyze the truture of optimal reommended trategie given a punihment length. The propertie haraterized in the following propoition an effetively redue the deign pae of the optimal reommended trategy given and thu redue the omputation omplexity of the optimal rating protool deign. Propoition 3. If we have that e > and a <, the optimal rating protool exhibit the following truture: (i) A -agent doe not reeive ervie from ome agent, i.e., (,) q = D, $ q Î. (ii) If ˆ (, q ) = F for ome ˆq, then agent with uffiiently high rating ore alway reeive ervie from - agent, i.e., (, q ) = F for all min{ln q ³ / ln b, b }. (iii) - agent reeive ervie from other agent whoe rating ore are uffiiently high, i.e., if q Î{,, -} atifie q ³ -(ln - ln Y( a, e, )) ln b, where b (-a) (-e) e-( -a) (-e) e Y( ae,, ) =, (9) + ( -a) ( - e) e+ a 3

14 then (, q ) = F. (iv) -agent alway provide ervie to other -agent, i.e., (, ) = F. Proof: See Appendix C. A Propoition 3 how, to ontrut an optimal rating protool, uffiient punihment hould be provided to agent with low rating ore while uffiient reward hould be provided to agent with high rating ore. C. Illutration with = and = We an repreent a reommended trategy a an ( + ) ( + ) matrix whoe (, ij-entry ) i given by (, ) i- j -. Propoition 3 provide ome truture of an optimal reommended trategy in the firt olumn and the lat row of the matrix repreentation, but it doe not fully haraterize the olution of DP. Here we aim to obtain the olution of DP for = and and analyze how it hange with the parameter. We firt begin with the ae of two rating, i.e., =. In thi ae, if (, q q ) = F for ome (, qq ), the relevant inentive ontraint to utain = (, ) i d( - e)[ v () - v ()] ³. By Propoition 3(ii) and (iv), if (, q q ) = F for ome ( qq, ), then (, ) = (, ) = F, provided that e > and a <. Hene, among the total of 6 poible reommended trategie, only four an be optimal reommended trategie. Thee four reommended trategie are éd Fù éf Fù éd Fù éd Dù 3 D =,,, F F = D F = = = D F D D. () êë úû êë úû êë úû êë úû For notational onveniene, we define a reommended trategy D by D (,) q = D for all q and D (, q q ) = F for all q and all q >. In (), we have 3 D =. The following propoition peifie the optimal reommended trategy given the parameter. Propoition. Suppoe that < (- ae ) < /. Then ìï b( -a) ( - e) e if <, b + b( - a) ( - e) e b( -a) ( - e) e b( -a)( - e)[ -( -a) e] if <, = ï í + b( -a) (- e) e b -b(-a) (- e) e 3 b( -a)( - e)[ -( -a) e] if < b( -a)( - e), -b(-a) (- e) e b ï if b( -a)( - e) < <. ïî b () i i Proof: et = (, ), for i =,, 3,. We obtain that U = ( -h () )( b- ), U = ( -h () h ())( b-), U = ( -h ())( b- ), U =. 3 ()

15 Sine < (- ae ) < /, we have h () < h (). Thu, we have U > U > U 3 > U. Alo, we obtain that h h 3() - 3() =, () - () =. v () - v () = ()( b-), v () - v () = b- ()( b-), v v b v v (3) Thu, we have v 3() - v 3() > v () - v () > v () - v () > v () - v (). By hooing the reommended trategy that yield the highet oial welfare among feaible one, we obtain the reult. Propoition how that the optimal reommended trategy i determined by the ervie ot-to-benefit ratio, / b. When / b i uffiiently mall, the reommended trategy an be utained, yielding the highet oial welfare among the four andidate reommended trategie. A / b inreae, the optimal reommended trategy hange from to to 3 and eventually to. Fig. 3 how the optimal reommended trategie with = a varie. The parameter we ue to obtain the reult in the figure of thi paper are et a follow unle otherwie tated: b =.8, a =., e =., and b =. Fig. 3(a) plot the inentive for ooperation of the four reommended trategie. We an find the region of in whih eah trategy i utained by omparing the inentive for ooperation with the ervie ot for,, and 3, and with - for. The olid portion of the line indiate that the trategy i utained while the dahed portion indiate that the trategy i not utained. Fig. 3(b) plot the oial welfare of the four andidate trategie, with olid and dahed portion having the ame meaning. The triangle-mared line repreent the optimal value, whih tae the maximum of the oial welfare of all utained trategie. Next, we analyze the ae of three rating, i.e., =. In order to provide a partial haraterization of the optimal reommended trategy, we introdue the following notation. et # be the reommended trategy with = that maximize min{ v () -v (), v () - v ()} among all the reommended trategie with =. et g d( - e) a defined in Appendix A, and define a reommended trategy by (,) - = D B and (, q q ) = F for all ( qq, ) ¹ ( -,) #. We have the following onluion about and. Propoition 5. Suppoe that e >, a <, and B B (i) # D = ; (ii) if h () < h (), then h () - g b < <. () b h () g B =. Proof: (i) et D = (, ). Then v ()- v () = v ()- v () = b. We an how that, under the given ondition, any hange from D reult in a dereae in the value of v () - v (), whih prove that D maximize min{ v () -v (), v () - v ()}. 5

16 (ii) Sine e > and a <, we have h () q > for all q =,,, and thu replaing D with F in an element of a reommended trategy alway improve oial welfare. Hene, we firt onider the reommended trategy F F defined by (, q q ) = F for all ( qq, ) F. maximize oial welfare U among all the reommended trategie with =, but v () - v () = v () - v () =. Thu, we annot find parameter uh that atifie the F inentive ontraint, and thu F ¹. Now onider reommended trategie in whih there i exatly one D element. We an how that, under the given ondition, having (, q q ) = D at (, qq ) uh that q > yield v v () - v () <, wherea having (, q q ) = D at ( qq, ) uh that q = yield both () v - v () > and () - v () >. Thu, for any reommended trategy having the only D element at ( qq, ) uh that q >, there do not exit parameter in the onidered parameter pae with whih the inentive ontraint for -agent, d( - e) é v () v () ù ê - ³ ë ú, i atified. On the other hand, for any reommended trategy having the only D û element at ( qq, ) uh that q =, we an atify both inentive ontraint by hooing b >, a <, e < /, and uffiiently loe to. Thi how that, among the reommended trategie having exatly one D element, only thoe having D in the firt olumn are poibly utainable. Sine h () < h () < h (), ahieve the highet oial welfare among the three andidate reommended trategie. et u try to better undertand now what the ondition in Propoition 5 mean. Propoition 5(i) implie that the maximum inentive for ooperation that an be ahieved with three rating i b( -a)( - e)b. Hene, ooperation an be utained with = if and only if b( -a)( - e)b ³. That i, if / b > b( -a)(- e), D then i the only feaible reommended trategy and thu U =. Therefore, when we inreae while holding other parameter fixed, we an expet that hange from D D to around = b( -a)( - e) b. Note that the ame i oberved with = in Propoition. We an ee that [ h ()/ h ()][(- g)/ g] onverge to a a goe to and b goe to. Hene, for given value of b,, and e, the ondition () i atified and thu ome ooperation an be utained if a and b are uffiiently loe to and, repetively. Conider a rating protool = (, ). We obtain that B t t B min{ v () -v (), v () - v ()} = v () - v () = ( -a) ( -e) e( b- b) (5) 3 and U = ( -( -a) ( -e) e )( b- ). Propoition 5(ii) i tating that B = when the ommunity ondition are favorable. More preiely, we have B = if ( -a) ( -e) e( b- b) ³, or 3 b( -a) ( - e)( -e) e. (6) b 3 + b ( - a) ( - e)( - e) e 6

17 3 Alo, Propoition 5(ii) implie that U ( -( -a) ( -e) e )( b- ) alway hold. In Fig., we how the optimal value and the optimal reommended trategy of DP a we vary. The optimal reommended trategy hange in the following order before beoming a inreae: D B éf F Fù éd F Fù éd F Fù 3 = D F F, F F F, D F F, = = êf F F F F F F F F ë ú û ê ë ú û ê ë ú û (7) éf F Fù éf F Fù éd F Fù éd F Fù = F F F, D F F, F F F, D F F. = = = êd F Fú êd F Fú êd F Fú êd F F ë û ë û ë û ë ú û Note that = for mall and 7 D = for large (but not too large to utain ooperation), whih are onitent with the diuion about Propoition 5. For the intermediate value of, only the element in the firt olumn hange in order to inreae the inentive for ooperation. We find that the order of the optimal reommended trategie between B = and 7 D = depend on the ommunity parameter ( bbae.,,,, ) V. EXTENSIONS A. Rating Sheme with Shorter Punihment ength So far we have foued on MPRS under whih any deviation in reported ation reult in the rating ore of. Although thi la of rating heme i imple in that a rating heme an be identified with the number of rating ore, it may not yield the highet oial welfare among all poible rating heme when there are report error. When there i no report error, i.e., e =, an agent maintain rating ore a long a he follow the preribed reommended trategy. Thu, in thi ae, punihment exit only a a threat and it doe not reult in an effiieny lo. On the ontrary, when e > and a <, there exit a poitive proportion of agent with rating to - in the tationary ditribution even if all the agent follow the reommended trategy. Thu, there i a tenion between effiieny and inentive. In order to utain a rating protool, we need to provide a trong punihment o that agent do not gain by deviation. At the ame time, too evere a punihment redue oial welfare. Thi obervation ugget that, in the preene of report error, it i optimal to provide inentive jut enough to prevent deviation. If we an provide a weaer punihment while utaining the ame reommended trategy, it will improve oial welfare. One way to provide a weaer punihment i to ue a random punihment. For example, we an onider a rating heme under whih the rating ore of a q -agent beome in the next period with probability q q Î (,] and remain the ame with probability - q q when he reportedly deviate from the reommended trategy. By varying the punihment probability q q for q -agent, we an adjut the everity of the punihment applied to q -agent. Thi la of rating heme an be identified by (,{ q q }). MPRS an be onidered a a peial ae where q q = for all q. 7

18 Another way to provide a weaer punihment i to ue a maller punihment length, denoted M. Under the rating heme with ( + ) rating ore and punihment length M, rating ore are updated by ìï min{ q +, } if a = ( q, q ), R tqq (,, a ) = ï R í ï max{ q - M, } if a ( q, q). R ¹ ïî (8) When a q -agent reportedly deviate from the reommended trategy, hi rating ore i redued by M in the next period if q ³ M and beome otherwie. Note that thi rating heme i analogou to real-world rating heme for redit rating and auto inurane ri rating. Thi la of rating heme an be identified by ( M, ) with M. MPRS an be onidered a a peial ae where M =. In thi paper, we fou on the eond approah to invetigate the impat of the punihment length on the oial welfare U and the inentive for ooperation min{ d( - e)[ v (min{ q +, }) -v (max{ q- M, })]} of a q rating protool, whih i now defined a ( M,, ). The punihment length M affet the evolution of the rating ditribution, and the tationary ditribution of rating ore with the rating heme ( M, ), { h(, ) ( q )} q=, atifie the following equation: M h () = ( -a) e h ( q), ( M, ) ( M, ) q= h () q = (-a)(-e) h ( q- ) + (- a) eh ( q + M) for q -M, ( M, ) ( M, ) ( M, ) h () q = (-a)(-e) h ( q-) for - M + q -, ( M, ) ( M, ) M å h ( ) = (-a)(- e){ h ( ) + h ( - )} + a. ( M, ) ( M, ) ( M, ) (9) { m ( q )} = be the umulative ditribution of { h (, ) ( )} M q q =, i.e., m (, ) () q = h (, ) () M å for M et ( M, ) q q =,,. Fig. 5 plot the tationary ditribution { h(, ) ( q )} q= and it umulative ditribution { m(, ) ( q )} q= M for = 5 and M =,, 5. We an ee that the umulative ditribution monotonially dereae with M, i.e., m () q m () q for all q if M > M. Thi how that, a the punihment length inreae, there are more ( M, ) ( M, ) agent holding a lower rating ore. A a reult, when the ommunity adopt a reommended trategy that treat an agent with a higher rating ore better, inreaing the punihment length redue oial welfare while it inreae the inentive for ooperation. Thi trade-off i illutrated in Fig. 6, whih plot oial welfare and the inentive for C ooperation under a rating protool (3, M, ) for M =,, 3, where the reommended trategy 3 C (, q q ) = F if and only if q ³ q, for all q. q = M C i defined by In general, the reommended trategy adopted in the ommunity i determined together with the rating heme in order to maximize oial welfare while atifying the inentive ontraint. The deign problem with variable We an further generalize thi la by having the punihment length depend on the rating. That i, when a q -agent reportedly deviate from the reommended trategy, hi rating i redued to q - M q in the next period for ome M q q. 8

19 punihment length an be formulated a follow. Firt, note that the expeted period payoff of a q -agent, v () q, an be omputed by (6), with the modifiation of the tationary ditribution to { h(, ) ( q )}. Agent long-term q= payoff an be obtained by olving (8), with the tranition probabilitie now given by ìï ' - e if q = q + ï ' min{, }, ' p ( q q) = ï if max{ M, }, for all íe q = q- q ÎQ. (3) ï otherwie, ïî Finally, the deign problem an be written a max imize U = h ( q ) v ( q ) ( M,, ) å q ( M, ) ubjet to d( - e)[ v (min{ q +, }) -v (max{ q- M, })] ³, " q uh that $ q uh that ( q, q ) = F, d( - e)[ v (min{ q +, }) -v (max{ q-m, })] ³-, " q uh that ( q, q ) = D " q. We find the optimal reommended trategy given a rating heme ( M, ) for = 3 and M =,, 3, and plot the oial welfare and the inentive for ooperation of the optimal reommended trategie in Fig. 7. Sine different value of M indue different optimal reommended trategie given the value of, there are no monotoni relationhip between the punihment length and oial welfare a well a the inentive for ooperation, unlie in Fig. 6. The optimal punihment length given an be obtained by taing the punihment length that yield the highet oial welfare, whih i plotted in Fig. 8. We an ee that, a the ervie ot inreae, the optimal punihment length inreae from to to 3 before ooperation beome no longer utainable. Thi reult i intuitive in that larger require a tronger inentive for ooperation, whih an be ahieved by having a larger punihment length. B. Whitewah-Proof Rating protool So far we have retrited our attention to rating heme where newly joining agent are endowed with the highet rating ore, i.e., K =, without worrying about the poibility of whitewahing. We now mae the initial rating ore K a a hoie variable of the deign problem while auming that agent an whitewah their rating ore in order to obtain rating ore K [7]. At the end of eah period, agent an deide whether to whitewah their rating ore or not after oberving their rating ore for the next period. If an agent hooe to whitewah hi rating ore, then he leave and re-join the ommunity with a fration of agent and reeive initial rating ore K. The ot of whitewahing i denoted by ³. w The inentive ontraint in the deign problem (8) are aimed at preventing agent from deviating from the preribed reommended trategy. In the preene of potential whitewahing attempt, we need additional inentive ontraint to prevent agent from whitewahing their rating ore. A rating protool i whitewah-proof if and M (3) 9