Predictive decision-making mechanisms based on off-line and on-line reasoning

Transcription

1 Swinburne University of Technology Melbourne Predictive decision-making mechanisms based on off-line and on-line reasoning A dissertation submitted in satisfaction of the requirements for the degree Doctor of Philosophy in Faculty of Information and Communication Technologies by Jakub Wojciech Brzostowski 2007

2 To my friends... who have always helped me to survive the hard times i

3 Abstract Negotiation is an interaction allowing for solving conflicts between multiple parties. The process of negotiation involves different types of decisions. Before the interaction starts the agent needs to choose its partner, and during the interaction the agent needs to decide on what offers to propose and what offers to accept. The outcome of these decisions may be improved when the agent uses forecasting, through the learning and reasoning techniques, of prospective partners behaviour. In this thesis two predictive decision-making mechanisms are proposed. The first mechanism allows for the selection of negotiation partners, and the second allows for the generation of negotiation offers during the actual negotiation. The selection mechanism uses the memory of previous encounters to make prediction about the potential behaviour of prospective negotiation partner. The prediction is obtained in a form of probability or possibility distribution for each potential negotiation partner, assigning to each potential negotiation outcome a degree of belief that this outcome will occur in the current negotiation. The distribution corresponding to the modelled agent is then fused with the utility function of the decision-maker which yields the value of expected utility. The agents maximizing the value of expected utility are considered the best prospective partners. The second decision-making mechanism uses the sequence of offers of both negotiating parties, up to the current negotiation stage, to predict the concession curve of the opponent. This prediction takes the form of a function mapping potential future offers of the modelling agent into the next offers of its counterpart. The construction of the concession curve forecast employs the regression analysis. The agent fits the memory of offers to each of the possible models of negotiation behaviour. The model that fits best to the data is then chosen for ii

4 the forecasting. The forecast obtained in such a manner is used to generate offers during the remaining negotiation rounds. The most important contributions of this dissertation to the subject matter are the new application of the concept of case-based reasoning, the adaptation of the case-based reasoning mechanism to the negotiation context, the new algorithm for the estimation of the possibilistic expected utility, the application of the regression analysis to the forecasting of the multi-tactic negotiation strategy and the discussion of the limitations of the regression analysis. Both decision-making mechanisms are validated experimentally. iii

5 Declaration I declare that this thesis contains no material which has been accepted for the award to the candidate of any other degree or diploma, except where due reference is made in the text of the thesis; to the best of my knowledge contains no material previously published or written by another person except where due reference is made in the text of the thesis, and where the work is based on joint research or publications, discloses the relative contributions of the respective workers or authors. I acknowledge that this thesis has been professionally edited. The editing has addressed only the style and grammar of the thesis and not its substantive content. Jakub Wojciech Brzostowski iv

6 Publications 1. Zaynab Raeesy, Jakub Brzostowski, and Ryszard Kowalczyk, (2007),Towards a Fuzzy Linguistic Model for Human-Like Multi-Agent Negotiation., The 2007 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2-5 November 2007, Silicon Valley, USA (accepted for publication 25 July 2007) 2. Mohan B Chhetri, Suk Keong Goh, James Lin, Jakub Brzostowski, and Ryszard Kowalczyk, (2007), Agent-based Negotiation of Service Level Agreements for Web Service Compositions., Joint Conference of the INFORMS Section on Group Decision and Negotiation, the EURO Working Group on Decision and Negotiation Support, and the EURO Working Group on Decision Support Systems (GDN 07), May 2007, Montreal, Canada, Group Decision and Negotiation 2007, Volume II, pp Jakub Brzostowski, Mohan B Chhetri, and Ryszard Kowalczyk, (2007), Three decision-making mechanisms for the negotiation agents., Joint Conference of the INFORMS Section on Group Decision and Negotiation, the EURO Working Group on Decision and Negotiation Support, and the EURO Working Group on Decision Support Systems, (GDN 07), May 2007, Montreal, Canada, Group Decision and Negotiation 2007, Volume II, pp Jakub Brzostowski, Ryszard Kowalczyk (2007). On fuzzy projection-based utility decomposition in compound multi-agent negotiations. World Congress Of The International Fuzzy Systems Association (IFSA 07), Cancun, Mexico, Foundations of Fuzzy Logic and Soft Computing, pp v

7 5. Jakub Brzostowski, Ryszard Kowalczyk (2006). Negotiation partners selection mechanism based on context-dependent similarity relations. The 6th International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2007), Honolulu, Hawai i, USA. 6. Jakub Brzostowski, Ryszard Kowalczyk (2006). Adaptive negotiation with on-line prediction of opponent behaviour in agent-based negotiations. The 2006 IEEE / WIC / ACM International Conference on Intelligent Agent Technology, Hong-Kong, China, IEEE, pp Jakub Brzostowski, Ryszard Kowalczyk (2006). Experimental evaluation of possibilistic mechanism for negotiation partners selection. The 2nd International Workshop on Rational, Robust, and Secutre Negotiations in Multi-Agent Systems (RRS2006), Hakodate, Japan. 8. Mohan B. Chhetri, Jian Ying Zhang, Jakub Brzostowski, Suk. K. Goh, Jian Lin, Boris Wu and Ryszard Kowalczyk (2006). Experimentation with three different approaches of agent-based negotiation. The Workshop on Service-Oriented Computing and Agent-Based Engineering (SOCABE 2006), Hakodate, Japan. 9. Jakub Brzostowski, Ryszard Kowalczyk (2005). Predicting partner s behaviour in agent negotiation. The 5th International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2006), Hakodate, Japan, ACM, pp vi

8 10. Jakub Brzostowski, Ryszard Kowalczyk (2005). Modelling Partner s Behaviour in Agent Negotiation The 18th Joint Australian Conference on Artificial Intelligence (AI 2005), Sydney, Australia, in Lecture Notes in Artificial Intelligence Lecture Notes in Artificial Intelligence by Springer, pp Jakub Brzostowski, Ryszard Kowalczyk (2005). Efficient algorithm for estimation of qualitative expected utility in possibilistic case-based reasoning. 21st Conference on Uncertainty in Artificial Intelligence (UAI 2005), Edinburgh, Scotland, AUAI Press, pp Jakub Brzostowski, Ryszard Kowalczyk (2005). On Possibilistic Case-based Reasoning for Selecting Partners for Multi-attribute Agent Negotiation. The 4th International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2005), Utrecht, Netherlands, ACM, pp Jakub Brzostowski, Ryszard Kowalczyk (2004). On Possibilistic Case-based Reasoning for Selecting Partners in Multi-Agent Negotiation. The 17th Joint Australian Conference on Artificial Intelligence (AI 2004), Cairns, Australia, in Lecture Notes in Artificial Intelligence (LNAI) by Springer, pp Peter Braun, Jakub Brzostowski, Gregory Kersten, Jinbaek Kim, Ryszard Kowalczyk, Stefan Strecker, Rustam Vahidov (2006): e-negotiation Systems and Software Agents: Method, Models, and Applications. In Jatinder Gupta, Guisseppi Forgionne, Manuel Mora (Eds.): Intelligent Decision-Making Support Systems (i-dmss): Foundations, Applicatons, and Challenges. Springer-Verlag London, Chapter 15, pp vii

9 15. Jakub Brzostowski, Ryszard Kowalczyk (2005). Possibilistic Case-based Reasoning for Selecting Partners in Multi-attribute Multi-Agent Negotiations In: O.Hryniewicz, J. Kacprzyk, J. Koronacki, S.T. Wierzchon (Eds.), Issues in Intelligent Systems. Paradigms. EXIT, Warszawa, pp 1 17 (book chapter). viii

10 Table of Contents 1 Introduction Background and preliminaries Selection of agents for interaction Approaches for selecting the interaction partners Negotiation supported with different techniques Preliminaries The negotiation thread Evaluation decisions The basic decision-making models for negotiation Learning and reasoning in negotiation Research questions The mechanism for selecting negotiation partners Problem description and approach Preliminaries The notion of α-cut Pareto order and monotonicity of multi-dimensional function The notion of Pareto frontier The concept of case-based reasoning and case-based problem solving ix

11 4.2.5 Expected utility theory Case-based decision theory and expected utility theory Case-based selection of negotiation partners The summary of the case-based selection mechanism Simple scenario description The similarity and pseudo-similarity relations The case-based reasoning for negotiation outcome prediction The representation of possibility distribution in a discrete form The representation of possibility distribution by the use of α-cuts Transformation of the distribution density function into cumulative distribution The transformation of the function in discrete representation The transformation of the function in the α-cut representation The Pareto frontier of the α-cut Qualitative expected utility Efficient algorithm for estimation of expected utility Example of calculations The calculations in discrete representation The calculations in the α-cut representation The simulation of multi-agent system x

12 4.13 Results and discussion First scenario Second scenario Third scenario Summary of the chapter The negotiation offers generation mechanism Problem description and approach Preliminaries - the approach of regression for the time series forecasting Gauss-Newton approach Newton s approach Levenberg-Marquardt approach An example of application of iterative regression algorithm The models of negotiation partners behaviour The limitations of the regression analysis in the negotiation context The strategy with polynomial time-dependent part The strategy with exponential time-dependent part The regression approach for the concession curve forecasting The determination of concession based on the regression forecast Results and discussion The scenario with full overlap of zones of acceptance and the same deadline of both parties xi

13 5.8.2 The scenario with full overlap of zones of acceptance and the deadline of adaptive agent shorter than deadline of static strategy agent The scenario with full overlap of zones of acceptance and the deadline of adaptive agent higher than deadline of static strategy agent The scenario with medium overlap of zones of acceptance and the same deadlines of both negotiation parties The scenario with medium overlap of zones of acceptance and the deadline of the adaptive agent shorter than the deadline of the static agent The scenario with medium overlap of zones of acceptance and the deadline of the adaptive agent longer than the deadline of the static agent The scenario with small overlap of zones of acceptance and the same deadlines of both parties The scenario with small overlap of zones of acceptance and the deadline of the adaptive agent shorter than the deadline of the static agent The scenario with small overlap of zones of acceptance and the deadline of the adaptive agent longer than the deadline of the static agent Summary of the chapter Summary of results, conclusions, answers to research questions and further work xii

14 6.1 Answers to research questions Negotiation partners selection mechanism Negotiation offers generation mechanism Further work References xiii

15 List of Figures 2.1 Examples of curves generated according to polynomial decision function (left picture) and the exponential one (right picture). [27] Problem definition The illustration of α-cut U 0.6 = [ 0.65, 0.65] of the function u(x) Leak s model of CBR according to [45] Illustration of a similarity based distribution obtained from a case base of three cases: (q 1, a, o 1 ), (q 2, a, o 2 ), (q 3, a, o 3 ) The illustration of a distribution obtained with extended inference rule The illustration of the selection mechanism The illustration of the reasoning for the history with two cases α-cut M α for two-dimensional space as a union of rectangles Illustration of transformation of density of possibility distribution to possibility distribution for one-dimensional decision space α-cut Π α for two-dimensional space and its Pareto frontier marked with circles The illustrating example showing the aggregation of distribution with utility function in one dimensional space. The set P is interval P = [0.4, 0.5] and set O consists of one point O = {0.5} Illustration for lemma 2 in one-dimensional space D = [0, 1] Illustration for lemma 3 in two-dimensional space D = [0, 1] Illustration for lemma 4 in two-dimensional space D = [0, 1] xiv

16 4.15 Density of possibility distribution for the first agent and its cumulative possibility distribution Density of possibility distribution for the second agent and its cumulative possibility distribution The function π 1 ν for possibility distribution The function π 2 ν for possibility distribution Density of probability distribution for the first agent and its cumulative probability distribution Density of probability distribution for the second agent and its cumulative probability distribution The function π 1 ν for probability distribution The function π 2 ν for probability distribution Example of calculations for the first agent. The α-cuts of both functions Example of calculations for the second agent. The α-cuts of both functions The illustration of the simulated multi-agent system with 3 buyeragents and 3 seller-agents. The dotted arrows indicate the current contracts The relation between the number of cases n c used for reasoning and the error of prediction for four different types of mechanisms The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection xv

17 4.28 The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (the selection of two agents) The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (two agents selection) The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (the selection of three agents) The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (three agents selection) The relation between the number of cases n c used for reasoning and the error of prediction for four different types of mechanisms The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) 98 xvi

18 4.36 The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (two agents selection) The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (two agents selection) The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (three agents selection) The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (three agents selection) The relation between the number of cases n c used for reasoning and the error of prediction for four different types of mechanisms The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (twoagent selection) xvii

19 4.44 The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (two-agent selection) The relation between the number of cases n c used for reasoning and the average gain of utility when the random selection was substituted with the possibilistic and probabilistic selection (three agents selection) The relation between the number of cases n c used for reasoning and the degrees of statistical significance of utility gain (t-values) (three-agent selection) The illustration of fitting the model f to the sequence of 7 observations from the example given above The illustration of the iterative regression algorithm in the space of two estimation parameters θ = (θ 1, θ 2 ). The black curves represent contours of the residual sum of squares S( θ). The algorithm starts in point θ a and moves to next estimation θ a+1. After four iterations the algorithm converges to the minimum of the function S( θ) The illustration of accuracy of the forecast for different values of β and k parameters for the first regression approach and for 8 offers as input (in terms of the percentage of acceptance zone). The highlighted area indicates the cases where the accuracy was lower than the 10% of the acceptance zone xviii

20 5.4 The illustration of accuracy of the forecast for different values of β and k parameters for the second regression approach and for 8 offers as input (in terms of the percentage of acceptance zone). The highlighted area indicates the cases where the accuracy was lower than the 10% of the acceptance zone The illustration of accuracy of the forecast for different values of β and k parameters for the first regression approach and for 12 offers as input (in terms of the percentage of acceptance zone). The highlighted area indicates the cases where the accuracy was lower than the 10% of the acceptance zone The illustration of accuracy of the forecast for different values of β and k parameters for the second regression approach and for 12 offers as input (in terms of the percentage of acceptance zone). The highlighted area indicates the cases where the accuracy was lower than the 10% of the acceptance zone The illustration of the negotiation process The illustration of the consecutive concessions Illustration of different negotiation threads leading to the same negotiation outcome The illustration of nine possible scenarios with different levels of overlaps of the zones of acceptance and different deadlines Scenario 1: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using polynomial decision function) xix

21 5.12 Scenario 1: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent using polynomial decision function) Scenario 1: The comparison of the negotiation lengths in the cases of the modelling approach and the average of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent using polynomial decision function) Scenario 1: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using exponential decision function) Scenario 1: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent using exponential decision function) Scenario 1: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Sample encounters with the BLA, BM A and BSA strategies (polynomial decision functions, client modelling provider) Sample encounters with the LSR strategies (polynomial decision functions, client modelling provider and provider modelling client) 145 xx

22 5.19 Sample encounters with the BLA, BM A and BSA strategies (exponential decision functions, client modelling provider) Scenario 2: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) Scenario 2: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 2: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 2: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) Scenario 2: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) xxi

23 5.25 Scenario 2: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Scenario 3: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) Scenario 3: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 3: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 3: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) Scenario 3: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) xxii

24 5.31 Scenario 3: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Scenario 3: Sample encounters with the A) BLA, B) BMA strategies - failures of adaptive mechanism and C) BSA - low utility gain of the adaptive mechanism (exponential decision functions, client modelling provider) Scenario 4: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) Scenario 4: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 4: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 4: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) xxiii

25 5.37 Scenario 4: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) Scenario 4: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Scenario 4: Sample encounters with the BLA, BM A and BSA strategies (polynomial decision functions, client modelling provider) Scenario 5: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) Scenario 5: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 5: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) xxiv

26 5.43 Scenario 5: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) Scenario 5: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) Scenario 5: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Sample encounters. First two figures show the success and failure in the encounter with the BLA strategy. The third figure shows the success in the encounter with BM A strategy (polynomial decision functions, client modelling provider) Sample encounters. First figure shows the failure in the encounter with the BMA strategy. The third and fourth figure show the success and failure in the encounter with BSA strategy (polynomial decision functions, client modelling provider) Scenario 6: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) xxv

27 5.49 Scenario 6: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 6: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 6: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) Scenario 6: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) Scenario 6: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) Sample encounters. First figure shows the failure in the encounter with the BLA and BM A strategies. (polynomial decision functions, client modelling provider) xxvi

28 5.55 Sample encounters. The first figure shows the failure in the encounter with the CLA strategy (A). The second shows the success with the CMA strategy (B). The third shows the failure with the CM A strategy (C). The fourth shows the encounter against CSA strategy (D). (polynomial decision functions, client modelling provider) Scenario 7: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using polynomial decision function) Scenario 7: The comparison of the modelling approach with the average of all the static strategies and the best static strategy (provider-agent modelling client-agent using polynomial decision function) Scenario 7: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 7: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using exponential decision function) xxvii

29 5.60 Scenario 7: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent using exponential decision function) The illustration of encounters against the strategies BLA (A) and LLA (B), polynomial decision function, client modelling provider The illustration of encounters against the strategies CLA (A), CM A (B) and CSA (C), (polynomial decision function, client modelling provider) The illustration of encounters against the strategies BLA (A) and LLA (B) (exponential decision function, client modelling provider) The illustration of encounters against the strategies BM A (A), BSA (B), LM A (C) and LSA (D), exponential decision function, client modelling provider The illustration of encounters against the strategies CSA strategy, success (A) and failure (B), exponential decision function, client modelling provider Scenario 8: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using polynomial decision function) xxviii

30 5.67 Scenario 8: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent using polynomial decision function) Scenario 8: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 8: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent using exponential decision function) Scenario 8: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent using exponential decision function) Scenario 8: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 9: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with polynomial decision function) xxix

31 5.73 Scenario 9: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with polynomial decision function) Scenario 9: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, polynomial decision function) Scenario 9: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (client-agent modelling provider-agent with exponential decision function) Scenario 9: The comparison of the modelling approach with the average gain of all the static strategies and the best static strategy (provider-agent modelling client-agent with exponential decision function) Scenario 9: The comparison of the negotiation lengths in the cases of the modelling approach and the average gain of all the static strategies (the scenarios with both roles profiles: client-agent modelling provider-agent and provider-agent modelling client-agent, exponential decision function) The illustration of encounters against strategies CM A (A) and CSA (B), polynomial decision function, client modelling provider. 174 xxx

32 5.79 The illustration of encounters against strategies LLA (A), LM A (B) and LSA (C), polynomial decision function, client modelling provider The illustration of encounters against strategies BLA (A), BM A (B) and BSA(C, exponential decision function, client modelling provider) The illustration of encounters against strategies CLA (A), CM A (B) and CSA (C), exponential decision function, client modelling provider xxxi

33 List of Tables 4.1 Example of history with two cases and the current situation. The value s i 1 is the description of the negotiation strategy (the β parameter) and the values s i 2 and s i 3 are the boundaries of the zone of acceptance (utility function description), the value of o i is the obtained value of negotiated attribute and τ i is the resulting utility from the viewpoint of buyer-agent The timings for both approaches. The timing is presented for the old and new approach. The column att illustrates the number of negotiation attributes. The column sum is the value of compound time needed by the approach based on discretization and column estim illustrates the timing of the efficient algorithm Example of history with five cases and the current situation for the first agent Example of history with five cases and the current situation for the second agent The n observations. x i is the i-th input vector consisting of two values (x 1 i, x 2 i ), y i is the output value An example of 7 observations. t i is the i-th time point, y i is the observed value in time point t i The computation of residuals r i for all observations (t i, y i ) and the residual sum of squares S( θ a ) The n observations. t i is the i-th time point, y i is the observed value in time point t i xxxii

34 5.5 The values of residuals r i for all observations (t i, y i ) depending on the estimation θ a = (0, 5, 6, 2) The negotiation thread xxxiii

35 CHAPTER 1 Introduction Nowadays, many large scale systems operate in ever-changing, highly open dynamic environment. These interconnected systems create open distributed systems. Rachmurn [56] gives some examples of the open distributed systems, such as: the Grid [29], peer-to-peer computing [58], the semantic web [6], web services [65], e-business [43], m-commerce [70, 72], autonomic computing [42] and pervasive computing environments [61]. The ever growing size and complexity of such systems lead to many important and difficult problems. Theses problems can be tackled by means of the agent technology. The characteristic feature of open distributed systems is their modularity, which means they consist of multiple components located usually in different places and responsible for performing different tasks [38, 68]. The building components of the system have to interact with each other in many different ways to achieve the common goal of the whole system and the goals of particular modules. The agent technology is a promising tool when applied to distributed systems [38]. Each component of the system can be integrated with an agent that represents particular component of the system. The role of such an agent is to act on behalf of the component to achieve the objective. Agents have to interact in different ways because of dependencies occurring between represented components. These interactions sometimes lead to conflicts that have to be resolved. The automated negotiation is the usual way of solving these conflicts [38]. There are many problems in the process of designing an appropriate decisionmaking mechanism for automated negotiation. The mechanism is dependent on the particular context and the object of negotiation. Before the negotiation starts, the negotiation agent has to choose the negotiation partners because the set of all candidates may be large and therefore the negotiation with all of them may 1

36 be impractical or infeasible. Limited knowledge of the environment and of the negotiation participants are the essential problems encountered by agents during the negotiation. Among these problems is the lack of knowledge of preferences, both one s own and negotiation partner. It is very difficult to elicit one s own preferences before the negotiation starts, especially in the case of multi-attribute negotiation. The usual lack of knowledge of the preferences of the negotiation partners is also a difficult problem but it is, at least partially, solvable by the use of learning and reasoning techniques. Another problem is the choice of one s own negotiation strategy and again the lack of knowledge of the partners negotiation behaviour might lead to poor negotiation outcomes. In this thesis we propose two decision-making mechanisms supporting the negotiation agents. The first one allows for selecting the best prospective negotiation partners based on their behaviour in previous encounters. The second one allows for on-line learning about the behaviour of the negotiation partner that improves the negotiation outcomes with this partner. The systems mentioned above consist of various entities whose interests have to be represented by some actors in order to handle all possible interactions between these entities. As mentioned at the beginning the agents may represent various components of the open distributed system that interact with each other in flexible ways in order to achieve its design objectives in complex and dynamic environment. The following definition of an agent will be used in this thesis: Definition 1 (Wooldridge and Jennings [73]) An agent is a computer system situated in an environment and capable of flexible autonomous action in this environment in order to meet its design objectives. Considering this definition some features of an agent can be distinguished: reactivity the ability to perceive the changes in the environment, including the actions of other agents, and to respond to these changes, proactiveness the ability to satisfy its goals by taking advantage of all the opportunities, 2

37 social ability the ability to cooperate with other agents in the environment in order to achieve the goals, both individual and common, adaptivity the ability to learn and improve through experience. Agents representing various components of a distributed system interact with each other creating a so-called multi-agent system. The nature of interaction may vary. It may be just an exchange of information between the agents. In some cases the agents have to coordinate their actions in order to achieve goals and the agents have sometimes to resolve their conflicts. We focus in this thesis on the last type of interaction, i.e. the conflict resolution. The interactions can be divided into two groups, namely the competitive and cooperative interactions. Competitive interactions in such interactions agents try to satisfy their own goals, e.g. conflict solutions satisfying preferences encoded by utility functions. Utility functions assign to each potential solution a score called utility value. The agent is trying to maximize the value of utility in the interaction with partners. Such behaviour of an agent is sometimes called selfish or self-interested. The selfish behaviour usually involves performing a sequence of actions, while perceiving changes in the environment and other agents. The agent acts taking into consideration the knowledge acquired during the entire interaction up to the point of making decision. Typically, in competitive interaction, an agent does not reveal its own preferences and potential negotiation strategy since opposite, selfish agent could take advantage of such a knowledge. Cooperative interactions in these types of interactions agents submit their own preferences to the interested parties. These parties can be either interaction partners or the centre of the system (broker, mediator, auctioneer, etc.). The preferences are fused and a solution is found that maximizes an aggregate of preferences of all interested parties. The aim of a cooperative interaction is to maximize a social welfare, usually meaning the maximization of the sum of utilities of all participants. The agent partially sacrifices its goals in order to make possible the satisfaction of the aggregated preferences of the whole group 3

38 of participants. It does not exclude the satisfaction of individual goals but this satisfaction level of an individual goal might be partial. The focus of this thesis is on a particular type of competitive interaction called positional bargaining. In such an interaction the agents exchange the proposal of potential solutions until they find a solution satisfying preferences of all interested parties. Every next proposal or counter-proposal has a lowered utility value compared to the previous one. In this sense, agents propose concessions to their counterparts in order to decrease the gap between the offers of both negotiating parties in terms of the utility value. In positional bargaining this gap is constantly reduced by the actions of the negotiators until they reach a consensus. Such a type of negotiation allows the agents to reveal the information about their preferences partially in each stage of the encounter. The agents concede from the most preferred value of negotiation attribute towards the lowest acceptable value. The agreement will be usually some point located in the range of acceptable solutions. Agents using the positional bargaining encounter a number of problems. Firstly, the negotiating parties might have different deadlines. This inconsistency may result in a broken negotiation because one agent usually does not know that the counterpart wants to end the encounter earlier and the consensus is not yet reached. Additionally, the overlaps of the sets of acceptable solutions of the negotiating parties might be sometimes small, which decreases the chance of reaching agreement. Because of the usual roles of the negotiating parties, such as seller and buyer, the preferences conflict, which means the better the solution is for one party the worse it is for its counterpart. The conflicting preferences also make it difficult to reach agreement. The limited knowledge of the opponents type, for instance the lack of knowledge about a partners reservation value, strongly influences the negotiation outcome. Most of the problems in competitive bargaining might be partially solved by applying the learning and reasoning mechanisms by the negotiation agent. These mechanisms can either help to avoid breaking the negotiation by one of the parties or to improve the negotiation outcome. Both the selection of negotiation partner and the generation of offers during the negotiation can be supported by predictive decision-making mechanisms. In the case of partners selection an agent can reason from previous negotiations which 4