Problem restructuring for better decision making in recurring decision situations

This paper proposes the use of restructuring information about choices to improve the performance of computer agents on recurring sequentially dependent decisions. The intended situations of use for the restructuring methods it defines are website platforms such as electronic marketplaces in which agents typically engage in sequentially dependent decisions. With the proposed methods, such platforms can improve agents’ experience, thus attracting more customers to their sites. In sequentially-dependent-decisions settings, decisions made at one time may affect decisions made later; hence, the best choice at any point depends not only on the options at that point, but also on future conditions and the decisions made in them. This “problem restructuring” approach was tested on sequential economic search, which is a common type of recurring sequentially dependent decision-making problem that arises in a broad range of areas. The paper introduces four heuristics for restructuring the choices that are available to decision makers in economic search applications. Three of these heuristics are based on characteristics of the choices, not of the decision maker. The fourth heuristic requires information about a decision-makers prior decision-making, which it uses to classify the decision-maker. The classification type is used to choose the best of the three other heuristics. The heuristics were extensively tested on a large number of agents designed by different people with skills similar to those of a typical agent developer. The results demonstrate that the problem-restructuring approach is a promising one for improving the performance of agents on sequentially dependent decisions. Although there was a minor degradation in performance for a small portion of the agents, the overall and average individual performance improved substantially. Complementary experimentation with people demonstrated that the methods carry over, to some extent, also to human decision makers. Interestingly, the heuristic that adapts based on a decision-maker’s history achieved the best results for computer agents, but not for people.

[1]  Nicholas R. Jennings,et al.  Socially intelligent reasoning for autonomous agents , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[2]  B. E. Eckbo,et al.  Appendix , 1826, Epilepsy Research.

[3]  C. Starmer.  Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk , 2000 .

[4]  Louis Anthony Cox,et al.  Wiley encyclopedia of operations research and management science , 2011 .

[5]  Avshalom Elmalech,et al.  Less is more: restructuring decisions to improve agent search , 2011, AAMAS.

[6]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[7]  Ya'akov Gal,et al.  A cultural sensitive agent for human-computer negotiation , 2012, AAMAS.

[8]  David Sarne,et al.  Advanced service schemes for a self-interested information platform , 2014, AAMAS.

[9]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[10]  D. Kahneman.  Thinking, Fast and Slow , 2011 .

[11]  Sarit Kraus,et al.  The Search for Coalition Formation in Costly Environments , 2003, CIA.

[12]  Andrew Martin,et al.  Better Heuristics for Economic Search — Experimental and Simulation Evidence , 1990 .

[13]  Amos Azaria,et al.  A system for advice provision in multiple prospectselection problems , 2013, RecSys.

[14]  P. Todd,et al.  Making Good Decisions with Minimal Information: Simultaneous and Sequential Choice , 2001 .

[15]  Amos Azaria,et al.  Combining psychological models with machine learning to better predict people’s decisions , 2012, Synthese.

[16]  Michael D. Lee,et al.  A Hierarchical Bayesian Model of Human Decision-Making on an Optimal Stopping Problem , 2006, Cogn. Sci..

[17]  Panos Markopoulos,et al.  Means based adaptive persuasive systems , 2011, CHI.

[18]  R. Radner,et al.  Decision and Organization. A Volume in Honor of Jacob Marschak. Edited by C.B. Mc Guire and Roy Radner. Studies in Mathematical and Managerial Economics, volume 12. Amsterdam, London, North-Holland Publishing Company, 1972 X p. 361 p., fl. 63.00. , 1973, Recherches économiques de Louvain.

[19]  J. Bakos.  Reducing buyer search costs: implications for electronic marketplaces , 1997 .

[20]  Sarit Kraus,et al.  Managing parallel inquiries in agents' two-sided search , 2008, Artif. Intell..

[21]  E. H. Simpson,et al.  The Interpretation of Interaction in Contingency Tables , 1951 .

[22]  Edmund S. Yu,et al.  Evolving and messaging decision-making agents , 2001, AGENTS '01.

[23]  D. Ariely.  Predictably Irrational: The Hidden Forces That Shape Our Decisions , 2008 .

[24]  Samuel P. M. Choi,et al.  Optimal Time-Constrained Trading Strategies for Autonomous Agents , 2000 .

[25]  Gideon Keren,et al.  Additional tests of utility theory under unique and repeated conditions , 1991 .

[26]  Douglas E Levy,et al.  A 2-phase labeling and choice architecture intervention to improve healthy food and beverage choices. , 2012, American journal of public health.

[27]  Douglas H. Wedell,et al.  Evaluations of Single‐ and Repeated‐Play Gambles , 2011 .

[28]  Douglas H. Wedell,et al.  Contemplating Single versus Multiple Encounters of a Risky Prospect , 1994 .

[29]  Peter Haddawy,et al.  Toward Case-Based Preference Elicitation: Similarity Measures on Preference Structures , 1998, UAI.

[30]  Harri Oinas-Kukkonen,et al.  Behavior Change Support Systems: A Research Model and Agenda , 2010, PERSUASIVE.

[31]  Sarit Kraus,et al.  Mass programmed agents for simulating human strategies in large scale systems , 2007, AAMAS '07.

[32]  Adam Burgess,et al.  ‘Nudging’ Healthy Lifestyles: The UK Experiments with the Behavioural Alternative to Regulation and the Market , 2012, European Journal of Risk Regulation.

[33]  William P. Dommermuth.  The Shopping Matrix and Marketing Strategy , 1965 .

[34]  M. Allais.  The Foundations of a Positive Theory of Choice Involving Risk and a Criticism of the Postulates and Axioms of the American School (1952) , 1979 .

[35]  P. Samuelson.  Risk and uncertainty: a fallacy of large numbers , 1963 .

[36]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[37]  D A Redelmeier,et al.  Discrepancy between medical decisions for individual patients and for groups. , 1990, The New England journal of medicine.

[38]  Sarit Kraus,et al.  Resolving crises through automated bilateral negotiations , 2008, Artif. Intell..

[39]  G. Kalyanaram,et al.  Nudge: Improving Decisions about Health, Wealth, and Happiness , 2011 .

[40]  R. Thaler,et al.  Gambling with the house money and trying to break even: the effects of prior outcomes on risky choice , 1990 .

[41]  Thomas S. Ferguson,et al.  Who Solved the Secretary Problem , 1989 .

[42]  Ya'akov Gal,et al.  Training with automated agents improves people's behavior in negotiation and coordination tasks , 2014, Decis. Support Syst..

[43]  Israel Spiegler,et al.  Modeling the search for the least costly opportunity , 2009, Eur. J. Oper. Res..

[44]  Andrew Schotter,et al.  Real-Time Search in the Laboratory and the Market , 2009 .

[45]  Gary L. Lilien,et al.  Chapter 2 Explanatory and predictive models of consumer behavior , 1993, Marketing.

[46]  Sarit Kraus,et al.  The influence of social dependencies on decision-making: initial investigations with a new game , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[47]  S. Mullainathan,et al.  Do People Mean What They Say? Implications for Subjective Survey Data , 2001 .

[48]  Yale M. Braunstein,et al.  ECONOMIC SEARCH: AN EXPERIMENTAL STUDY , 1981 .

[49]  Trevor J. M. Bench-Capon,et al.  Autonomous Agents and Multi-agent Systems Manuscript No. Using Argumentation to Model Agent Decision Making in Economic Experiments , 2022 .

[50]  Henry Montgomery,et al.  Gambling decisions and information about expected value. , 1982 .

[51]  Tai Gyu Kim,et al.  When Things Don't Add Up , 2005, Psychological science.

[52]  Clare Harries,et al.  Measuring Doctors' Self-insight into their Treatment Decisions , 2000 .

[53]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[54]  Sarit Kraus,et al.  Reaching Agreements Through Argumentation: A Logical Model and Implementation , 1998, Artif. Intell..

[55]  J. P. Kline,et al.  Individual differences in anterior EEG asymmetry between high and low defensive individuals during a rumination/distraction task , 2005 .

[56]  Willem A. Wagenaar,et al.  Violation of utility theory in unique and repeated gambles , 1987 .

[57]  Pablo Noriega,et al.  A Framework for Argumentation-Based Negotiation , 1997, ATAL.

[58]  D. Kahneman,et al.  Timid choices and bold forecasts: a cognitive perspective on risk taking , 1993 .

[59]  Amos Azaria,et al.  Automated Strategies for Determining Rewards for Human Work , 2012, AAAI.

[60]  Sarit Kraus,et al.  Modeling agents based on aspiration adaptation theory , 2012, Autonomous Agents and Multi-Agent Systems.

[61]  John Roberts,et al.  Development and Testing of a Model of Consideration Set Composition , 1991 .

[62]  Jeffrey O. Kephart,et al.  Shopbot Economics , 2004, Autonomous Agents and Multi-Agent Systems.

[63]  Sarit Kraus,et al.  Programming agents as a means of capturing self-strategy , 2008, AAMAS.

[64]  Melvyn Sim,et al.  TRACTABLE ROBUST EXPECTED UTILITY AND RISK MODELS FOR PORTFOLIO OPTIMIZATION , 2009 .

[65]  T. Wallsten,et al.  Individual Decision Behavior , 1972 .

[66]  Ya'akov Gal,et al.  Facilitating the Evaluation of Automated Negotiators using Peer Designed Agents , 2010, AAAI.

[67]  Sarit Kraus,et al.  Comparing Agents' Success against People in Security Domains , 2011, AAAI.

[68]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[69]  M. Weitzman.  Optimal search for the best alternative , 1978 .

[70]  Amos Azaria,et al.  Strategic Information Disclosure to People with Multiple Alternatives , 2011, AAAI.

[71]  Harry M. Markowitz,et al.  Mean-variance approximations to expected utility , 2014, Eur. J. Oper. Res..

[72]  Pattie Maes,et al.  Kasbah: An Agent Marketplace for Buying and Selling Goods , 1996, PAAM.

[73]  David Carmel,et al.  Exploration and Adaptation in Multiagent Systems: A Model-based Approach , 1997, IJCAI.

[74]  Daniel Schunk,et al.  The Relationship Between Risk Attitudes and Heuristics in Search Tasks: A Laboratory Experiment , 2009 .

[75]  Ramesh Sharda,et al.  Decision Support Systems , 2009, Handbook of Automation.

[76]  Abhijit Chaudhury,et al.  An empirical investigation of decision-making satisfaction in web-based decision support systems , 2004, Decis. Support Syst..

[77]  M. Rabin.  Psychology and Economics , 1997 .

[78]  Thomas T. Hills,et al.  Information Search in Decisions From Experience , 2010, Psychological science.

[79]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[80]  J. Sonnemans,et al.  Strategies of search , 1998 .

[81]  David Sarne,et al.  Information Sharing Under Costly Communication in Joint Exploration , 2013, AAAI.

[82]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[83]  Martin Weber,et al.  Prospect Theory, Mental Accounting, and Differences in Aggregated and Segregated Evaluation of Lottery Portfolios , 2001, Manag. Sci..

[84]  Avshalom Elmalech,et al.  Search More, Disclose Less , 2013, AAAI.

[85]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[86]  Roger A. Drake.  Processing Persuasive Arguments: 2. Discounting of Truth and Relevance as a Function of Agreement and Manipulated Activation Asymmetry , 1993 .

[87]  Stuart M. Shieber,et al.  Agent decision-making in open mixed networks , 2010, Artif. Intell..

[88]  Colin Camerer,et al.  Risk and time preferences: linking experimental and household survey data from Vietnam , 2010 .

[89]  Thomas Klueter,et al.  Is There an Ebay for Ideas? Insights from Online Knowledge Marketplaces , 2011 .

[90]  Lisa R. Klein,et al.  Consumer search for information in the digital age: An empirical study of prepurchase search for automobiles , 2003 .

[91]  J Ormel,et al.  Factors affecting contrasting results between self-reported and performance-based levels of physical limitation. , 1996, Age and ageing.

[92]  David Sarne,et al.  The effectiveness of peer-designed agents in agent-based simulations , 2012, Multiagent Grid Syst..

[93]  John D. Hey,et al.  Search for rules for search , 1982 .

[94]  J. Quiggin.  A theory of anticipated utility , 1982 .

[95]  G. L. S. Shackle,et al.  Decision Order and Time in Human Affairs , 1962 .

[96]  Elaine M. Liu.  Time to Change What to Sow: Risk Preferences and Technology Adoption Decisions of Cotton Farmers in China , 2013, Review of Economics and Statistics.

[97]  Avshalom Elmalech,et al.  Evaluating the Applicability of Peer-Designed Agents in Mechanisms Evaluation , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[98]  M. Rehm,et al.  Proceedings of AAMAS , 2005 .

[99]  Michael L. Littman,et al.  Algorithms for Sequential Decision Making , 1996 .

[100]  Alexandre Padilla.  Review of Richard H. Thaler and Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happiness , 2009 .

[101]  Martin Weber,et al.  Investment Decisions and Time Horizon: Risk Perception and Risk Behavior in Repeated Gambles , 2005, Manag. Sci..

[102]  Marco Guerini.  Toward ethical persuasive agents , 2005 .

[103]  G. Shafer,et al.  Expected Utility Hypotheses and the Allais Paradox. , 1982 .

[104]  R. Hogarth,et al.  BEHAVIORAL DECISION THEORY: PROCESSES OF JUDGMENT AND CHOICE , 1981 .

[105]  P. Schoemaker.  The Expected Utility Model: Its Variants, Purposes, Evidence and Limitations , 1982 .

[106]  Amos Azaria,et al.  Automated Agents' Behavior in the Trust-Revenge Game in Comparison to Other Cultures (Extended Abstract) , 2014 .

[107]  James B. Thomas,et al.  The value of action and inference in dynamic decision making , 1987 .

[108]  V. I. Danilov,et al.  Expected utility theory under non-classical uncertainty , 2010 .

[109]  A. Colman,et al.  Ambiguity aversion in the long run: Repeated decisions under risk and uncertainty , 2009 .

[110]  Nikita Joshi,et al.  The Art of Choosing , 2016 .

[111]  David Sarne,et al.  Increasing threshold search for best-valued agents , 2010, Artif. Intell..

[112]  S. Iyengar.  The Art of Choosing , 2010 .

[113]  I. Erev,et al.  Small feedback‐based decisions and their limited correspondence to description‐based decisions , 2003 .

[114]  David Sarne,et al.  Competitive Shopbots-Mediated Markets , 2013, TEAC.

[115]  Sarit Kraus,et al.  How to Change a Group's Collective Decision? , 2013, IJCAI.