Comparison of competing market mechanisms with reinforcement learning in a carpooling scenario

Abstract In this paper a multi-agent simulation was implemented to analyze the dynamics of different market mechanisms with a Reinforcement Learning algorithm in the context of a carpooling market. The agents in the simulation, car owners (COs) and non car owners (NCOs), had to sell or buy a car seat for multiple rounds by picking one of two possible mechanisms: Dutch Auction or Fixed Price. In the beginning of the simulation the agents have no information about the efficiency of these mechanisms and they are chosen with the same probability. In the course of the simulation a Reinforcement Learning algorithm alters the agents' preferences for the two mechanisms depending on their cumulative payoffs. The key finding is that sellers have a clear preference for the Dutch auction mechanism with differing degrees dependent on the seller/buyer ratio. Buyers on the other hand have no significant preference for any mechanism. If these results are replicable, they suggest that an increased utilization of the Dutch auction could lead to an expansion of the carpooling market, increasing its impact as an alternative means of transportation.

[1]  Martina Ziefle,et al.  Join the Ride! User Requirements and Interface Design Guidelines for a Commuter Carpooling Platform , 2013, HCI.

[2]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[3]  T. Pénard,et al.  What Drives Pricing Behavior in Peer-to-Peer Markets? Evidence from the Carsharing Platform BlaBlaCar , 2016, Information Economics and Policy.

[4]  Davy Janssens,et al.  Multi-agent simulation of individual mobility behavior in carpooling , 2014 .

[5]  Ruqu Wang.  Auctions versus Posted-Price Selling: The Case of Correlated Private Valuations , 1998 .

[6]  Klaus Kultti,et al.  Equivalence of Auctions and Posted Prices , 1999 .

[7]  Thomas Pitz,et al.  An Extended Reinforcement Algorithm for Estimation of Human Behaviour in Experimental Congestion Games , 2007, J. Artif. Soc. Soc. Simul..

[8]  Eric Guerci,et al.  Learning to bid in sequential Dutch auctions , 2014 .

[9]  J. Cross.  A Stochastic Learning Model of Economic Behavior , 1973 .

[10]  J. Rochet,et al.  Platform competition in two sided markets , 2003 .

[11]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[12]  Thomas Pitz,et al.  Transferring decisions to an algorithm: A simple route choice experiment , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[13]  Marc Rysman.  The Economics of Two-Sided Markets , 2009 .

[14]  Ruqu Wang.  Auctions versus Posted-Price Selling , 1993 .

[15]  Richard S. Sutton,et al.  Learning and Sequential Decision Making , 1989 .

[16]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[17]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[18]  W. Arthur.  Designing Economic Agents that Act Like Human Agents: A Behavioral Approach to Bounded Rationality , 1991 .

[19]  Atakelty Hailu,et al.  Are Auctions More Efficient Than Fixed Price Schemes When Bidders Learn? , 2004 .

[20]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[21]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[22]  D. Ricardo.  On the Principles of Political Economy and Taxation , 1891 .

[23]  S. Shaheen,et al.  Online and App-Based Carpooling in France: Analyzing Users and Practices—A Study of BlaBlaCar , 2017 .

[24]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[25]  Neel Sundaresan,et al.  Auctions versus Posted Prices in Online Markets , 2018, Journal of Political Economy.

[26]  Luk Knapen,et al.  Agent-Based Modeling for Carpooling , 2014 .

[27]  J. Rochet,et al.  Two-sided markets: a progress report , 2006 .

[28]  Teck-Hua Ho,et al.  Experience-Weighted Attraction Learning in Coordination Games: Probability Rules, Heterogeneity, and Time-Variation. , 1998, Journal of mathematical psychology.

[29]  W. Brian Arthur,et al.  On designing economic agents that behave like human agents , 1993 .

[30]  Thomas Pitz,et al.  Experiments and Simulations on Day-to-Day Route Choice-Behaviour , 2003, SSRN Electronic Journal.

[31]  C. Harley.  Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[32]  Sebastian J. Goerg,et al.  Learning in Experimental 2 X 2 Games , 2011 .

[33]  P. Klemperer.  Auction Theory: A Guide to the Literature , 1999 .

[34]  Sebastian J. Goerg,et al.  Learning in experimental 2×2 games , 2012, Games Econ. Behav..

[35]  Eric Guerci,et al.  An agent-based model for sequential Dutch auctions , 2013, 2013 Winter Simulations Conference (WSC).