AI-MIX: Using Automated Planning to Steer Human Workers Towards Better Crowdsourced Plans

One subclass of human computation applications are those directed at tasks that involve planning (e.g. tour planning) and scheduling (e.g. conference scheduling). Interestingly, work on these systems shows that even primitive forms of automated oversight on the human contributors helps in significantly improving the effectiveness of the humans/crowd. In this paper, we argue that the automated oversight used in these systems can be viewed as a primitive automated planner, and that there are several opportunities for more sophisticated automated planning in effectively steering the crowd. Straightforward adaptation of current planning technology is however hampered by the mismatch between the capabilities of human workers and automated planners. We identify and partially address two important challenges that need to be overcome before such adaptation of planning technology can occur: (i) interpreting inputs of the human workers (and the requester) and (ii) steering or critiquing plans produced by the human workers, armed only with incomplete domain and preference models. To these ends, we describe the implementation of AI-MIX, a tour plan generation system that uses automated checks and alerts to improve the quality of plans created by human workers; and present a preliminary evaluation of the effectiveness of steering provided by automated planning.

[1]  Jim Blythe,et al.  Task learning by instruction in tailor , 2005, IUI.

[2]  Been Kim,et al.  Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior , 2013, AAAI.

[3]  Krzysztof Z. Gajos,et al.  Human computation tasks with global constraints , 2012, CHI.

[4]  Dana S. Nau,et al.  SHOP2: An HTN Planning System , 2003, J. Artif. Intell. Res..

[5]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.

[6]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[7]  Hector Geffner,et al.  Probabilistic Plan Recognition Using Off-the-Shelf Classical Planners , 2010, AAAI.

[8]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[9]  Daniel Borrajo,et al.  From Unstructured Web Knowledge to Plan Descriptions , 2011, Information Retrieval and Mining in Distributed Environments.

[10]  Haim Kaplan,et al.  Answering Planning Queries with the Crowd , 2013, Proc. VLDB Endow..

[11]  Walter S. Lasecki,et al.  Real-Time Collaborative Planning with the Crowd , 2012, AAAI.

[12]  Qiang Yang,et al.  Learning action models from plan examples using weighted MAX-SAT , 2007, Artif. Intell..

[13]  Subbarao Kambhampati,et al.  Herding the Crowd: Automated Planning for Crowdsourced Planning , 2013, HCOMP.

[14]  Edith Law,et al.  Towards Large-Scale Collaborative Planning: Answering High-Level Search Queries Using Human Computation , 2011, AAAI.

[15]  Matthias Scheutz,et al.  Planning for human-robot teaming in open worlds , 2010, TIST.

[16]  Lydia B. Chilton,et al.  Cobi: communitysourcing large-scale conference scheduling , 2013, CHI Extended Abstracts.

[17]  Been Kim,et al.  Inferring Team Task Plans from Human Meetings: A Generative Modeling Approach with Logic-Based Prior , 2015, J. Artif. Intell. Res..

[18]  James F. Allen,et al.  TRAINS-95: Towards a Mixed-Initiative Planning Assistant , 1996, AIPS.

[19]  Tova Milo,et al.  CrowdPlanr: Planning made easy with crowd , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[20]  Subbarao Kambhampati,et al.  AI-MIX : How a Planner Can Help Guide Humans Towards a Better Crowdsourced Plan , 2014 .

[21]  Subbarao Kambhampati,et al.  Model-Lite Case-Based Planning , 2012, AAAI.