Bayesian Learning based Negotiation Agents for Supporting Negotiation with Incomplete Information

Transcription

1 ayesan Learnng base Negoaon Agens for upporng Negoaon wh Incomplee Informaon Jeonghwan Gwak an Kwang Mong m Absrac An opmal negoaon agen shoul have capably for mamzng s uly even for negoaon wh ncomplee nformaon n whch he agens o no know onen s prvae nformaon such as reserve prce (R an ealne To suppor negoaon wh ncomplee negoaon, hs work focuses on esgnng learnng agens calle ayesan learnng (L base negoaon agens (LNA aopng a me-epenen negoaon moel In LNA, L s use for esmang onen s R an he corresponng ealne s compue usng he esmae R LNA also has he capably of mamzng s uly usng esmae ealne nformaon when s ealne s shorer han or equal o he esmae ealne To evaluae he performance of LNA, LNA s compare o an agen wh complee nformaon an an agen wh ncomplee nformaon Emprcal resuls showe ha LNA acheve: always hgher uly han he agen wh ncomplee nformaon for all cases, 2 close o or slghly lower uly han he agen wh complee nformaon when s ealne s lower an equal o onen s ealne, an 3 lower uly han he agen wh complee nformaon when ealne s hgher han onen s ealne Ine Terms Auomae negoaon, negoaon agens, nellgen agens, ayesan learnng, negoaon wh ncomplee nformaon A I INTRODCTION TOMATED negoaon s efne as a process for resolvng fferences an conflcng goals among neracng agens There are wo ypes of negoaon envronmen, one for agens wh complee negoaon sengs an he oher for agens wh ncomplee negoaon sengs Whle agens wh complee nformaon know onen prvae nformaon such as reserve prce an ealne, agens wh ncomplee nformaon o no know onen prvae nformaon For negoaon wh complee nformaon, opmal soluons, for an agen can be easly eermne usng he neracng agen s prvae nformaon, eg, [] an [2] However, s no easy for an agen wh ncomplee nformaon o acheve opmal soluons For opmal soluons n negoaon wh ncomplee nformaon, a learnng meho s requre In hs work, ayesan learnng Manuscrp receve January 2, 20 Ths work was suppore by he Korea Research Founaon Gran fune by he Korean Governmen (MET (KRF D00092 Jeonghwan Gwak s wh he chool of Informaon an Communcaons (IC, Gwangju Insue of cence an Technology, ouh Korea (phone: ; fa: ; e-mal: gwak@gsackr Kwang Mong m s wh he chool of Informaon an Communcaons (IC, Gwangju Insue of cence an Technology, ouh Korea (phone: ; fa: ; e-mal: prof_sm_2002@yahoocom (L s aope for supporng negoaon wh ncomplee nformaon by fnng an onen s prvae nformaon There are some relae works [], [2], [3] usng L for supporng negoaon wh ncomplee nformaon In [3], L framework was frs nrouce o suppor negoaon In [] an [2], LGAN uses he synergy of L an a genec algorhm (GA A Conrbuon of hs work The fferences beween he above relae works an hs work are as follows: Compare o [] an [2], hs work suggess a new L meho for esmang an onen s reserve prce (R by esgnng a new cononal probably of L 2 nce esmaon of he eac R usng L s no possble, here es some esmaon errors of R To compensae for he esmaon errors, a GA was use n [2] for makng raeoff beween an agen s proposal an s onen proposal However, negoaon usng he GA can be fnshe quckly whou reachng opmal resuls Even hough he agen aopng a GA-base raeoff algorhm can ncrease he negoaon success raes, shoul gve up some amoun of uly As a prmarly research repor, hs paper s focuse on enhancng L par only bu no focusng usng a GA-base raeoff algorhm 3 Alhough hs paper s base on [] an [2], here es several fferen pons Compare o [] an [2], hs work uses fferen equaons n generang proposals for boh learnng an no-learnng agens Whle esmaon of onen R an ealne n [] an [2] s carre ou separaely, we calculae onen ealne usng esmae onen R because esmaon of onen R an ealne s ner-epenen (see econ II-C Furhermore, he calculae onen ealne nformaon s use for generang proposals o ncrease s uly when he calculae onen ealne s longer han or equal o s ealne The paper s organze as follows The negoaon moel n hs work s escrbe n econ II, an he esgn of propose L-base negoaon agens (LNA are escrbe n econ III econ IV shows epermenal resuls an analyzes he performance The fnal secon conclues hs work wh summary an suggess fuure works

2 II NEGOTIATION MODEL A Tme-epenen Negoaon Moel In hs work, we conser blaeral negoaon beween wo self-nerese agens over an ssue, eg, prce, qualy of servce, ec The wo agens have conflcng roles such as seller ( an buyer ( The agens negoae by echangng proposals usng Rubnsen s alernang-offers proocol [4] The fe (no learnng agen {, } generaes proposals a a me roun, 0, as follows: ( I R I, ( where for an 0 for I an R s he nal prce (he mos favorable prce ha can affor an reserve prce (he leas favorable prce ha can affor of, respecvely s he ealne an, 0, s he me-epenen sraegy of The concesson behavor of s eermne by he values he me-epenen sraegy [5] an s classfe as follows: Conclaory (0 : makes larger concesson n earler negoaon rouns an smaller concessons n laer rouns 2 Lnear ( : makes a consan rae of concesson 3 Conservave ( : makes smaller concesson n earler negoaon rouns an larger concessons n laer rouns Le D be he even n whch fals o reach an agreemen The uly funcon of s efne as :[ I, R ] D [0,] such ha ( D 0 an for any [ I, R ], ( ( D ( s gven as follows: R ( umn ( umn R I, (2 where umn s he mnmum uly ha receves for reachng an agreemen a R For epermenal purpose, he value of u s se as 0 A R, ( R 0 ( D 0 mn For he blaeral negoaon n hs work, s assume ha sars he negoaon by makng proposals o The negoaon process beween he wo agens wll be ermnae: n makng an agreemen when an offer or a couner-offer s accepe, or 2 n a conflc when one of he wo agens ealnes s reache An agreemen s reache when one agen proposes a eal ha maches or ecees wha anoher agen asks for, e, ( ( or ( ( Opmal Negoaon raegy wh Complee Informaon The opmal sraegy of s efne as he sraegy ha mamzes he uly of a agreemen me T c Le c be he agreemen prce (e, c The mamum sraegy T c ensures ( ( for all T c c For negoaon wh complee nformaon beween an, [] an [2] prove he followng Theorems an 2 (refer [] an [2] for eale llusraons: Theorem ( : acheves mamal uly when aops he sraegy I R log I R Theorem 2 ( : acheves mamal uly when aops he sraegy R I log R I C Negoaon wh Incomplee Informaon For a negoaon wh ncomplee nformaon, f an agen can esmae onen s R an ealne eacly, he agen can fn an opmal sraegy usng s corresponng heorem beween Theorems an 2 The relaonshp beween R an ealne s gven as follows The formulas for calculang R n (3 an ealne n (4 are erve from ( I R I (3 I R I For he gven I,, an, he calculaon of R s relae wh an he calculaon of s relae wh R Hence, calculaon of R an s no separable bu closely relae wh each oher If we can esmae eher R or ealne, he oher can be calculae from he esmae varable usng (3 or (4 If s assume ha I s known ( can be easly assume f he frs proposal s I, can be calculae usng I an wo proposals wh fferen me rouns as follows: I ( R I (5 I ( R I (6 y vng (5 by (6, he followng equaon s acheve I I Fnally, he followng s calculae (4 I log, where 3, (7 I In summary, f I s known, eac can be calculae by (7 when 3 Then, f eher R or ealne s esmae, he oher can be compue usng (3 or (4 In hs work, we esmae R an he corresponng ealne calculaon s conuce usng (4 III AYEIAN LEARNING AED ROOED AROACH In hs secon, we wll escrbe he esgn of LNA o suppor negoaon wh ncomplee nformaon

3 A ayesan Learnng of onen s R Le he prce range be n [MIN, MAX ] An agen aopng L forms a se of hypoheses {H } of onen s R, where H (MAX MIN / NH an N H s he number of hypoheses The -h hypohess of an onen s R s efne as R an he esmae onen R s efne as R The followng relaon (oman knowlege beween an R s erve from (3 I R I, R ( where R ( s he scoun rao of an measure by R ( For eample, gven I = 5, R = 85, 50 an 5, he followng Fgs an 2 show smulaon resuls of s proposals an corresponng scounng rao a 0,,,, respecvely A 25, he resuls shows an R 5 25 ( Hence, 50 R an equals o R I rce Fg s proposals a 0,,, Rao R ( Fg 2 Dscounng rao R ( a 0,,, sng he formula n (8 represenng relaon beween ( (8 I an R, he cononal probably (lkelhoo srbuon s esgne as follows: ( R I R ( I ( R I R (, (9 where R ( s he esmae scounng rao a me roun To acheve accurae cononal probably n (9, s crucal o oban an approprae R ( R ( s obane by he followng formula: R ( (, (0 where s calculae by (7 when 3 Due o he same mensonaly an ffculy of esmang onen ealne as onen R, ( s he esmae onen ealne a me roun, an s calculae usng (4 from he esmae onen R a prevous me roun, R ( In some cases, ( R shoul be zero as follows For esmang onen R usng L, f R, hen ( R 0 because wll no generae proposals hgher han s R mlarly, for esmang onen R usng L, f R, hen ( R 0 because wll no generae proposals lower han s R When receves, he ayesan upang formula revses s belef abou onen R wh pror probably srbuon s efne as follows: N H R R ( R ( R ( R, ( ( ( where he pror probably ( R s efne as ( R ( R an nally, s assume unform srbuon over all hypoheses Fnally, he epece value of R a me roun, R (, s compue usng he followng formula R ( ( R R (2 L-base Negoaon Agens LNA generaes he ne proposal usng he followng formula [2]: ( R, (3 ( where for an 0 for Compare o (, he man fference s ha (3 reas he prevous proposal as s new nal prce a me roun nl now, all maerals for generang (opmal sraegy are sue an prepare The proceure for generang opmal sraegy s escrbe n Algorhm

4 e as LNA ar (L sage: Generang roposals usng L nformaon If me roun 3 Compue R ( as an average value n feasble prce range Compue usng (4 an R ( Compue usng Theorem or 2 wh R ( an Generae a proposal If me roun 2 usng (3 wh e ( R as R ( 2 for all If me roun 3 Compue ( usng R ( Compue usng (7 Compue R ( usng (0 Compue ( R usng (9 for all Compue ( R usng ( for all e ( R ( R Compue R ( usng (2 Compue usng (4 an R ( Compue usng Theorem or 2 wh R ( an Generae a proposal usng (3 wh ar 2: Generang roposals usng Dealne Informaon (Case : he case when sare he negoaon frs If an ˆ If sare he negoaon frs ( R an ( ( ( If Generae a proposal as Else, Generae a proposal as s R (Case 2: he case when onen sare he negoaon frs If an ˆ If onen sare he negoaon frs Accep a proposal Algorhm L-base Negoaon Agens Algorhm consss of wo pars ar for generang proposals usng L nformaon s he man par of he algorhm as scusse n econ III-A ar 2 s he proceure for generang proposals usng ealne nformaon If LNA can learn s ealne s lower han or equal o onen ealne eacly ( ˆ, can sll mamze s uly even hough reaches s ealne Depenng on whch agen sare he negoaon frs, ar 2 s ve no he followng wo cases: (Case If LNA sare he negoaon frs, here s sll room for o acheve s mamum uly a me roun Ths can be carre ou by makng he proposal as onen proposal a on he conon ha each ( R an ( ( ( However, because esmang he onen proposal wll be ffcul an here wll es errors n esmang he proposal usng L, LNA wll generae he proposal as he onen proposal a ( (Case 2 If onen sare he negoaon frs, wll acheve s mamum uly a me roun by no makng a proposal ( ( an accepng he onen proposal a me roun IV EXERIMENTAL RELT To evaluae he effecveness an performances of LNA emprcally, hree ypes of negoaon scenaros beween an were sue as n Table I Throughou he hree negoaon scenaros, s se as a fe an no-learnng agen n whch generaes proposals usng ( an a sraegy ranomly fe a sar of negoaon has hree ypes accorng o he followng scenaros TALE I THREE NEGOTIATION CENARIO cenaro Agen Agen Complee Complee nformaon Incomplee nformaon Incomplee Incomplee nformaon Incomplee nformaon Incomplee wh L Incomplee nformaon learns onen s R usng L Incomplee nformaon cenaro ( as he opmal agen: has complee nformaon abou an has ncomplee nformaon abou nce knows s R an ealne, generaes proposals by aopng s opmal sraegy usng Theorem The negoaon resul correspons o he bes-case scenaro ha LNA s argeng for cenaro 2 ( as he fe an no-learnng agen: oh an have ncomplee nformaon abou each oher oes no know s R an ealne an generae proposals usng a sraegy ranomly fe whn he possble sraegy range a he sar of negoaon The negoaon resul correspons o he wors-case scenaro cenaro 3 ( as LNA: oh an have ncomplee nformaon abou each oher However, generaes proposals by aopng esmae sraeges usng Theorem wh esmae s R usng L an he corresponng calculaon of ealne Furhermore, uses some ealne nformaon for generang proposals (see ar 2 n Algorhm A Epermenal engs The agens parameer sengs are summarze n Table II Inally, an s Is, Rs an ealnes were ranomly selece n he gven ranges a sar of negoaon Three ype of ealnes hor, M, an Long were use (hor, M, (M, M an (Long, M were use for comparng negoaon resuls wh respec o ealne effecs (e, wh fferen barganng avanage The represenaon (Dealne of, Dealne of means ses he frs elemen

5 an ses he secon elemen as her ealnes For eample, (hor, M means aops he hor ealne an aops M ealne In he epermens of (hor, M, (M, M an (Long, M, 000 ranom runs for each scenaro were carre ou, an n each run, agens use he same Is, Rs, ealnes an nal sraeges for all he scenaros In he ranom generaon of an s sraeges a he sar of negoaon, he probables of generang conclaory (conceng raply an conservave (conceng slowly sraeges are same arameer ype Mnmum possble prce (MIN Mamum possble prce (MAX R R TALE II AGENT ARAMETER ETTING ossble arameer values 00 [MIN + 5, MIN + (MAX MIN /2] [R + 0, MAX 5] I [R, MAX ] I [MIN, R ] ossble sraegy range [0002, 0] Dealne hor 25 M 50 Long 00 N H 00 Epermenal Resuls The followng four performance measures were use: uccess rae, 2 Falure ype, 3 Average negoaon roun, an 4 s normalze average uly uccess rae (R s efne as R = N success /N oal, where N success s he number of successful eals an N oal s he oal number of eals To enfy he reasons of negoaon falure, wo ypes of negoaon falure were consere: Type I an Type II o represen nfeasble eals ue o s nfeasble hgh sraegy sengs an 2 s wrong sraegy esmaon, respecvely Average negoaon roun (ANR s measure by he average number of negoaon rouns requre o reach an agreemen for all successful eals s normalze average uly (NA s efne as NA where ( fnal me roun fnal N success 0 fnal, N success ma s he fnal proposal a he fnal negoaon an ma s he mamum uly of TALE III RELT OF [HORT, MID] CAE (hor, M Complee Incomplee Incomplee wh L uccess rae Falure ype a - Type I Type I Average negoaon rouns 's normalze average uly a As has a hgher sraegy value (eg, 0, here s a hgher possbly ha he negoaon can fal o generae approprae proposals before he shorer ( s ealne reaches (e, s fnal proposal s lower han R a s ealne In hs case, canno make he successful negoaon an Type I couns he kn of negoaon falures TALE IV RELT OF [MID, MID] CAE (M, M Complee Incomplee Incomplee wh L uccess rae Falure ype Average negoaon rouns 's normalze average uly TALE V RELT OF [LONG, MID] CAE (Long, M Complee Incomplee Incomplee wh L uccess rae Falure ype b - Type II Type II Average negoaon rouns 's normalze average uly b If learns neac R an ealne havng hgher error raes, he negoaon can fal o generae approprae proposals before he shorer ( s ealne reaches (e, s fnal proposal s hgher han R a s ealne In hs case, canno make he successful negoaon an Type II couns he kn of negoaon falures C Analyss an Dscusson The goal of LNA s o acheve he resuls ha are close o he Complee scenaro ha shows he opmum resuls for he gven sengs nce n (hor, M an (M, M cases, has shorer han or equal o he ealne of, ar 2 n LNA algorhm wll have sgnfcan role o mprove he performance pecfcally, (Case 2 n ar 2 wll have effec n hs case because wll sar he negoaon frs In conras, n (Long, M case, ar wll have sgnfcan role for achevng goo performance Observaon - (hor, M Case: Incomplee wh L scenaro acheve very close resuls o he Complee scenaro Analyss: nce has a shorer ealne han, f appropraely learns s R ( R an compues s ealne ( ˆ n Incomplee wh L scenaro, wll propose s prevous proposal a he ealne an successful agreemen wll be mae a he shorer ealne (see ar 2-(Case n Algorhm As shown n Table III, n Complee scenaro, he negoaon was ermnae a me rouns 25 an acheve normalze average uly Even hough n

6 Incomplee scenaro, acheve normalze average uly 0466 a average me rouns 968, n Incomplee wh L scenaro acheve normalze average uly 0575 a average me rouns 2484 The values of Incomplee wh L scenaro are very close o Complee scenaro The resuls showe ha (he LNA can learn s onen s ( s R an ealne when has longer ealne han ( ˆ almos eacly n (hor, M case Observaon 2 - (M, M Case: Incomplee wh L scenaro acheve resuls ha are close o he Complee scenaro Analyss: nce an have he same ealne, f learns s R ( R an compues s ealne ( ˆ appropraely n Incomplee wh L scenaro, wll propose s prevous proposal a he ealne an successful agreemen wll be mae a he ealne (see ar 2-(Case n Algorhm As shown n Table IV, n Complee scenaro, he negoaon was ermnae a me rouns 50 an acheve normalze average uly Even hough n Incomplee scenaro, acheve normalze average uly a average me rouns 362, n Incomplee wh L scenaro acheve normalze average uly a average me rouns 483 The values of Incomplee wh L scenaro are close o Complee scenaro The resuls showe ha (he LNA can learn he onen s ( s R an ealne when s oes no have shorer han s ealne ( ˆ appropraely wh some errors n (M, M case Observaon 3 - (Long, M Case: Incomplee wh L scenaro acheve resuls no close o he Complee scenaro bu acheve resuls beer han Incomplee scenaro Analyss: nce has he longer ealne han, f appropraely learns s R ( R an compues s ealne ( ˆ appropraely n Incomplee wh L scenaro, wll make proposals usng he sraegy by Theorem wh R an ˆ As shown n Table V, n Complee scenaro, he negoaon was ermnae a me rouns 50 an acheve normalze average uly Alhough has longer ealne han, canno acheve he mamum uly Ths s because f sars negoaon frs, wll ece wheher accep s proposal or no In Incomplee scenaro, acheve normalze average uly a average me rouns 3783 In Incomplee wh L scenaro acheve normalze average uly 0727 a average me rouns 4233 The values of Incomplee wh L scenaro are no close o Complee scenaro bu hey are hgher han Incomplee scenaro The resuls show ha (he LNA can learn he onen s ( s R an ealne wh some errors n (Long, M case V CONCLION AND FTRE WORK In hs paper, we showe he performance of propose LNA by conserng he case ha one agen uses L an ealne nformaon for generang s ne proposal n he hree ealne combnaons, (hor, M, (M, M an (Long, M The performance showe ha: In (hor, M case, LNA can learn he onen ealne s longer han s onen almos eacly 2 In (M, M case, LNA can learn he onen ealne s no shorer han s onen wh some errors 3 In (Long, M case, LNA can learn he onen s R an ealne wh more hgher errors From he resuls o 3, we conclue LNA can suppor he negoaon wh ncomplee nformaon However, we nee o mprove he performance, especally n he case of (Long, M case, n erms of boh he success rae an uly Due o he space lmaon, we only consere one fe case of ealne (M case for There are sll pleny of ealne combnaons such as (hor, hor, (M, hor, (Long, hor, (hor, Long, (M, Long an (Long, Long o fgure ou he whole sysem performance of propose LNA Furhermore, alhough we only consere he case ha one agen ( learns onen s nformaon usng L an ealne nformaon, wll be neresng o analyze he case ha boh agens can learn each oher In our fuure work, we wll conser he above ssues Moreover, he research for supporng negoaon wh ncomplee nformaon s sll beng carre ou o ncrease he performance of LNA by ncorporang evoluonary algorhms n L sage of LNA REFERENCE [] K M m, Y Guo, an h, "Aapve barganng agens ha negoae opmally an raply" pp [2] K M m, Y Guo, an h, LGAN: ayesan learnng an genec algorhm for supporng negoaon wh ncomplee nformaon, IEEE Trans ys Man Cybern, Cybern, vol 39, no, pp 98-2, Feb 2009 [3] D Zeng, an K ycara, ayesan learnng n negoaon, In J Hum-Compu u, vol 48, no, pp 25-4, 998 [4] A Rubnsen, A barganng moel wh ncomplee nformaon abou me preferences, Economerca, vol 53, no 5, pp 5-72, ep 985 [5] K M m, Equlbra, pruen compromses,an he "wang" game, IEEE Trans ys Man Cybern, Cybern, vol 35, no 4, pp , Aug 2005