Affordances and Constraints of Intelligent Decision Support for Military Command and Control : Three Case Studies of Support Systems ; Möjligheter och begränsningar med intelligent beslutsstöd i militär ledning : Tre fallstudier av teknikstöd

We present results from a study on constructing and evaluating a support tool for the extraction of patterns in distributed decision -making processes, based on design criteria elicited from a study on the work process involved in studying such decision-making. Specifically, we devised and evaluated an analysis tool for C2 researchers who study simulated decision-making scenarios for command teams. The analysis tool used text clustering as an underlying pattern extraction technique and was evaluated together with C2 researchers in a workshop to establish whether the design criteria were valid and the approach taken with the analysis tool was sound. Design criteria elicited from an earlier study with researchers (open-endedness and transparency) were highly consistent with the results from the workshop. Specifically, evaluation results indicate that successful deployment of advanced analysis tools requires that tools can treat multiple data sources and offer rich opportunities for manipulation and interaction (open-endedness) and careful design of visual presentations and explanations of the techniques used (transparency). Finally, the results point to the high relevance and promise of using text clustering as a support for analysis of C2 data.

[1]  Dieter Pfoser Indexing the Trajectories of Moving Objects , 2002 .

[2]  V Rajaraman,et al.  As we may think , 1988 .

[3]  Yolanda Gil,et al.  INSPECT: an Intelligent System for Air Campaign Plan Evaluation based on EXPECT , 1996 .

[4]  James A. Hendler,et al.  UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning , 1994, AIPS.

[5]  Gheorghe Tecuci,et al.  Training and Using Disciple Agents: A Case Study in the Military Center of Gravity Analysis Domain , 2002, AI Mag..

[6]  Wil vanderAalst,et al.  Workflow Management: Models, Methods, and Systems , 2004 .

[7]  Peter A. Jarvis,et al.  PASSAT: A User-centric Planning Framework , 2002 .

[8]  H. Dreyfus Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian , 2007 .

[9]  E. Salas,et al.  Establishing the Boundaries of a Paradigm for Decision-Making Research , 1996, Hum. Factors.

[10]  Kenneth D. Forbus,et al.  Incorporating AI into Military Decision Making: An Experiment , 2003, IEEE Intell. Syst..

[11]  Alexander Kott,et al.  A Knowledge-Based Tool for Planning of Military Operations: The Coalition Perspective , 2002 .

[12]  Pat Langley,et al.  Case-Based Seeding for an Interactive Crisis Response Assistant , 1998 .

[13]  Amedeo Cesta,et al.  User Interaction with an Automated Solver: The Case of a Mission Planner , 2004, PsychNology J..

[14]  Yuval Shahar,et al.  The Asgaard project: a task-specific framework for the application and critiquing of time-oriented clinical guidelines , 1998, Artif. Intell. Medicine.

[15]  Ola Leifler Combining Technical and Human-Centered Strategies for Decision Support in Command and Control - The ComPlan Approach , 2008 .

[16]  David Woods,et al.  Communication of Intent in Military Command and Control Systems , 2000 .

[17]  Alon Y. Halevy,et al.  A Platform for Personal Information Management and Integration , 2005, CIDR.

[18]  Richard E. Hayes,et al.  Power to the Edge: Command, Control in the Information Age , 2003 .

[19]  Sture Hägglund Introducing expert critiquing systems , 1993, Knowl. Eng. Rev..

[20]  Dana S. Nau,et al.  SHOP2: An HTN Planning System , 2003, J. Artif. Intell. Res..

[21]  MAGDALINI EIRINAKI,et al.  Web mining for web personalization , 2003, TOIT.

[22]  Barry G. Silverman,et al.  Survey of expert critiquing systems: practical and theoretical frontiers , 1992, CACM.

[23]  David L. Kleinman,et al.  Enablers of Self-Synchronization for Network-Centric Operations: Design of a Complex Command and Control Experiment , 2001 .

[24]  L. Bertalanffy AN OUTLINE OF GENERAL SYSTEM THEORY , 1950, The British Journal for the Philosophy of Science.

[25]  Jih-Jeng Huang,et al.  Multidimensional data in multidimensional scaling using the analytic network process , 2005, Pattern Recognit. Lett..

[26]  Stanley M. Halpin The Human Dimensions of Battle Command: A Behavioral Science Perspective on the Art of Battle Command. , 1996 .

[27]  E. Hutchins Cognition in the wild , 1995 .

[28]  Jens Rasmussen,et al.  Deciding and Doing: Decision Making in Natural Context , 1993 .

[29]  Jianmin Wang,et al.  A novel approach for process mining based on event types , 2007, IEEE International Conference on Services Computing (SCC 2007).

[30]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[31]  Perry L. Miller Medical Plan-Analysis: The Attending System , 1983, IJCAI.

[32]  Alice M. Mulvehill,et al.  JADE: A Tool for Rapid Crisis Action Planning , 1999 .

[33]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[34]  S. G. Axline,et al.  Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. , 1975, Computers and biomedical research, an international journal.

[35]  Juha Takkinen,et al.  From information management to task management in electronic mail , 2002 .

[36]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[37]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[38]  Yuval Shahar,et al.  A Task-Specific Ontology for the Application and Critiquing of Time-Oriented Clinical Guidelines , 1997, AIME.

[39]  Nicholas Kushmerick,et al.  Wrapper Induction for Information Extraction , 1997, IJCAI.

[40]  Francesco Ricci,et al.  Interactive Case-Based Planning for Forest Fire Management , 2000, Applied Intelligence.

[41]  Mica R. Endsley,et al.  Using Goal Directed Task Analysis with Army Brigade Officer Teams , 2002 .

[42]  T. Sylverberg,et al.  Drawing on Paper Maps: Reliable On-line Symbol Recognition of Handwritten Symbols Using a Digital Pen and a Mobile Phone , 2007, 2007 2nd International Conference on Pervasive Computing and Applications.

[43]  Pattie Maes,et al.  Agents that reduce work and information overload , 1994, CACM.

[44]  D. Schoen Educating the reflective practitioner , 1987 .

[45]  Luciano Serafini,et al.  Contextual Representation and Reasoning with Description Logics , 2011, Description Logics.

[46]  Peter A. Jarvis,et al.  CODA: Coordinating Human Planners , 2001 .

[47]  Diego Calvanese,et al.  The Description Logic Handbook , 2007 .

[48]  Hyoil Han,et al.  Survey of semantic annotation platforms , 2005, SAC '05.

[49]  Terry Winograd,et al.  The Action Workflow Approach to Workflow Management Technology , 1993, Inf. Soc..

[50]  Patricia M. Jones Cooperative support for distributed supervisory control: requirements, issues, and an example from mission operations , 1993, IUI '93.

[51]  Paul Dourish,et al.  Freeflow: mediating between representation and action in workflow systems , 1996, CSCW '96.

[52]  Karen L. Myers Abductive Completion of Plan Sketches , 1997, AAAI/IAAI.

[53]  B. Brehmer Dynamic decision making: human control of complex systems. , 1992, Acta psychologica.

[54]  Barbara D. Klein,et al.  User evaluations of IS as surrogates for objective performance , 2000, Inf. Manag..

[55]  Johan Jenvald,et al.  Simulation as a tool for problem detection in rescue operation planning , 2005 .

[56]  Henrik Eriksson,et al.  Domain-specific knowledge management in a Semantic Desktop , 2009, I-KNOW 2009.

[57]  Berndt Brehmer,et al.  The Dynamic OODA Loop : Amalgamating Boyd ’ s OODA Loop and the Cybernetic Approach to Command and Control ASSESSMENT , TOOLS AND METRICS , 2005 .

[58]  Ian Horrocks,et al.  From SHIQ and RDF to OWL: the making of a Web Ontology Language , 2003, J. Web Semant..

[59]  H. Gadamer,et al.  The philosophy of Hans-Georg Gadamer , 1997 .

[60]  James F. Allen,et al.  Mixed-Initiative Dialogue Systems for Collaborative Problem-Solving , 2005 .

[61]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[62]  Stephen F. Smith,et al.  Comirem: an intelligent form for resource management , 2005, IEEE Intelligent Systems.

[63]  Laura G. Militello,et al.  Evaluating Air Campaign Plan Quality in Operational Settings I , 1996 .

[64]  Dik Gregory Delimiting Expert Systems , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[65]  T. Connolly,et al.  The reinvention of decision making. , 1993 .

[66]  Juan Fernández-Olivares,et al.  Bringing Users and Planning Technology Together. Experiences in SIADEX , 2006, ICAPS.

[67]  Pavlos Moraitis,et al.  A distributed decision support system for strategic planning , 1997, Decis. Support Syst..

[68]  Ian M. Neale,et al.  First generation expert systems: a review of knowledge acquisition methodologies , 1988, The Knowledge Engineering Review.

[69]  David W. Aha,et al.  NaCoDAE: Navy Conversational Decision Aids Environment , 1998 .

[70]  Alexander Kott,et al.  Toward practical knowledge-based tools for battle planning and scheduling , 2002, AAAI/IAAI.

[71]  Joe Marks,et al.  Human-Guided Simple Search , 2000, AAAI/IAAI.

[72]  Karen L. Myers Domain Metatheories: Enabling User-Centric Planning , 2000 .

[73]  William S. Havens,et al.  A Framework for Designing and Evaluating Mixed-Initiative Optimization Systems , 2005 .

[74]  Izak Benbasat,et al.  Empirical Research in Information Systems: The Practice of Relevance , 1999, MIS Q..

[75]  Nancy J. Cooke,et al.  Measuring Team Knowledge: A Window to the Cognitive Underpinnings of Team Performance , 2003 .

[76]  Ismailcem Budak Arpinar,et al.  Ontology-Driven Automatic Entity Disambiguation in Unstructured Text , 2006, SEMWEB.

[77]  Mark S. Ackerman,et al.  The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility , 2000, Hum. Comput. Interact..

[78]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[79]  Anders Arpteg,et al.  Intelligent Semi-Structured Information Extraction , 2005 .

[80]  Peter Szolovits,et al.  Artificial Intelligence in Medicine , 1982 .

[81]  Robert E. Kraut,et al.  Technology for intellectual teamwork: perspectives on research and design , 1990 .

[82]  Dean Allemang,et al.  The Semantic Web - ISWC 2006, 5th International Semantic Web Conference, ISWC 2006, Athens, GA, USA, November 5-9, 2006, Proceedings , 2006, SEMWEB.

[83]  David Mendonça,et al.  Decision support for improvisation in response to extreme events: Learning from the response to the 2001 World Trade Center attack , 2007, Decis. Support Syst..

[84]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[85]  M. Bélanger,et al.  A Decision Support System for CoA Selection , 2022 .

[86]  Kenneth Baclawski,et al.  A core ontology for situation awareness , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[87]  Mark Dredze,et al.  Activity-Centric Email: A Machine Learning Approach , 2006, AAAI.

[88]  F. Bookstein,et al.  Two Structural Equation Models: LISREL and PLS Applied to Consumer Exit-Voice Theory , 1982 .

[89]  Stephen J. Andriole,et al.  Handbook of Decision Support Systems , 1989 .

[90]  Sue Holwell,et al.  Information, Systems and Information Systems: Making Sense of the Field , 1998 .

[91]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[92]  Alexander Kott,et al.  A New Technique Enables Dynamic Replanning and Rescheduling of Aeromedical Evacuation , 1998, AI Mag..

[93]  Erik Hollnagel,et al.  Barriers And Accident Prevention , 2004 .

[94]  Hendrik Blockeel,et al.  Web mining research: a survey , 2000, SKDD.

[95]  Robert C. Rubel War-Gaming Network-Centric Warfare , 2001 .

[96]  Steffen Staab,et al.  The X-COSIM integration framework for a seamless semantic desktop , 2007, K-CAP '07.

[97]  Marie A. Bienkowski,et al.  Demonstrating the Operational Feasibility of New Technologies: The ARPI IFDs , 1995, IEEE Expert.

[98]  Delwyn N. Clark A Literature Analysis of the Use of Management Science Tools in Strategic Planning , 1992 .

[99]  Charles R. Schwenk Cognitive simplification processes in strategic decision-making : insights from behavioral decison theory and cognitive psychology / 947 , 1983 .

[100]  James F. Allen,et al.  TRAINS-95: Towards a Mixed-Initiative Planning Assistant , 1996, AIPS.

[101]  Yolanda Gil,et al.  PLANET: A Shareable and Reusable Ontology for Representing Plans , 2000 .

[102]  Fox Weasel CoRaven 1 Weasel : A Mixed-Initiative System to Assist in Military Planning , 2005 .

[103]  Aaron V. Cicourel,et al.  The integration of distributed knowledge in collaborative medical diagnosis , 1990 .

[104]  Yolanda Gil,et al.  EXPECT: Explicit Representations for Flexible Acquisition , 1995 .

[105]  Barry G. Silverman Critiquing Human Error: A Knowledge Based Human-Computer Collaboration Approach , 1997 .

[106]  L. Suchman Human-Machine Reconfigurations: Plans and situated actions (2nd edition). , 2007 .

[107]  Christer Carlsson,et al.  Past, present, and future of decision support technology , 2002, Decis. Support Syst..

[108]  Ganesh Ramakrishnan,et al.  Identification of class specific discourse patterns , 2008, CIKM '08.

[109]  Yolanda Gil,et al.  A Problem-Solving Method for Plan Evaluation and Critiquing , 1999 .

[110]  Volker Haarslev,et al.  Querying the Semantic Web with Racer + nRQL , 2004 .

[111]  George Boolos,et al.  Computability and logic , 1974 .

[112]  Rogier Woltjer,et al.  Functional Modeling of Constraint Management in Aviation Safety and Command and Control , 2009 .

[113]  Yolanda Gil,et al.  Plan Representation and Reasoning with Description Logics , 2003 .

[114]  Jihie Kim,et al.  Evaluating expert-authored rules for military reasoning , 2003, K-CAP '03.

[115]  Siegfried Handschuh,et al.  The NEPOMUK Project - On the way to the Social Semantic Desktop , 2007 .

[116]  Charles R. Schwenk Cognitive simplification processes in strategic decision‐making , 1984 .

[117]  Enrico Motta,et al.  An ontological formalisation of the planning task , 2004 .

[118]  Ramanathan V. Guha,et al.  SemTag and seeker: bootstrapping the semantic web via automated semantic annotation , 2003, WWW '03.

[119]  Nigel Shadbolt,et al.  User-induced links in collaborative tagging systems , 2009, CIKM.

[120]  Hassan Gomaa,et al.  Prototyping as a tool in the specification of user requirements , 1981, ICSE '81.

[121]  Janet Wiles,et al.  Use of an automatic content analysis tool: A technique for seeing both local and global scope , 2009, Int. J. Hum. Comput. Stud..

[122]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[123]  David L. Kleinman,et al.  Inducing Adaptation in Organizations: Concept and Experiment Design , 2004 .

[124]  Drew McDermott,et al.  Issues in the Development of Human-Computer Mixed Initiative Planning , 1994 .

[125]  Henrik Eriksson An Annotation Tool for Semantic Documents , 2007, ESWC.

[126]  Jack Park,et al.  IRIS: Integrate. Relate. Infer. Share , 2005, Semantic Desktop Workshop.

[127]  Barbara E. Engelhardt,et al.  Automated planning for the Modified Antarctic Mapping Mission , 2001, 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542).

[128]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[129]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[130]  M. Weiser,et al.  Hot topics-ubiquitous computing , 1993 .

[131]  Richard E. Hayes,et al.  Understanding Command and Control , 2011 .

[132]  Alex Fukunaga,et al.  Iterative Repair Planning for Spacecraft Operations Using the Aspen System , 2000 .

[133]  Bijan Fazlollahi,et al.  Evaluation of decisional guidance in decision support systems: an empirical study , 1995 .

[134]  Edward H. Shortliffe,et al.  Adapting a Consultation System to Critique User Plans , 1983, Int. J. Man Mach. Stud..

[135]  Jonathan Kies,et al.  User and task analysis for interface design , 1998 .

[136]  Patricia M. Jones,et al.  Cooperative problem solving in human-machine systems: theory, models, and intelligent associate systems , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[137]  John A. Bateman,et al.  On the relationship between ontology construction and natural language: a socio-semiotic view , 1995, Int. J. Hum. Comput. Stud..

[138]  Patrik Larsson,et al.  Automatic handling of Frequently Asked Questions using Latent Semantic Analysis , 2009 .

[139]  Stephen F. Smith,et al.  Mixed-Initiative Resource Management: The AMC Barrel Allocator , 2000, AIPS.

[140]  J. Lawson Command control as a process , 1981, IEEE Control Systems Magazine.

[141]  Erik Hollnagel,et al.  Joint Cognitive Systems: Foundations of Cognitive Systems Engineering , 2005 .

[142]  K. Eisenhardt Building theories from case study research , 1989, STUDI ORGANIZZATIVI.

[143]  Lucy Suchman Plans and situated actions: the problem of human-machine communication , 1987 .

[144]  Carol McCann,et al.  RE-CONCEPTUALIZING COMMAND AND CONTROL , 2002 .

[145]  John Levine,et al.  Generation of Multiple Qualitatively Different Plan Options , 1998, AIPS.

[146]  D. Nadeem From Philosophy and Mental-Models to Semantic Desktop Research : Theoretical Overview , 2007 .

[147]  A. F. Chalmers,et al.  What Is This Thing Called Science , 1976 .

[148]  Paul E. Lehner On the Role of Artificial Intelligence in Command and Control , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[149]  S. Mohammed,et al.  The Measurement of Team Mental Models: We Have No Shared Schema , 2000 .

[150]  Bonnie M. Muir,et al.  Trust Between Humans and Machines, and the Design of Decision Aids , 1987, Int. J. Man Mach. Stud..

[151]  Henrik Eriksson,et al.  A Model for Document Processing in Semantic Desktop Systems , 2008 .

[152]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[153]  Stefan Arnborg,et al.  Information awareness in command and control: precision, quality, utility , 2000, Proceedings of the Third International Conference on Information Fusion.

[154]  Stephen J. Andriole,et al.  Information Technology for Command and Control , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[155]  Magnus Morin Multimedia representations of distributed tactical operations , 2002 .

[156]  Diane H. Sonnenwald,et al.  Information behavior in dynamic group work contexts: interwoven situational awareness, dense social networks and contested collaboration in command and control , 2000, Inf. Process. Manag..

[157]  M. Heidegger The Question Concerning Technology and Other Essays , 1977 .

[158]  David W. Aha,et al.  HICAP: An Interactive Case-Based Planning Architecture and its Application to Noncombatant Evacuation Operations , 1999, AAAI/IAAI.

[159]  Karol G. Ross,et al.  The Recognition-Primed Decision Model , 2004 .

[160]  Neville A. Stanton,et al.  Cognitive Task Analysis: Current use and practice in the UK Armed Forces and elsewhere , 2005 .

[161]  Kenneth D. Forbus,et al.  AI on the battlefield: an experimental exploration , 2002, AAAI/IAAI.

[162]  D. W. Hands,et al.  Against Method: Outline of an Anarchistic Theory of Knowledge , 1977 .

[163]  S. Mohammed,et al.  Team Mental Model: Construct or Metaphor? , 1994 .

[164]  Magnus Morin,et al.  How Modeling and Simulation Can Support MEDEVAC Training , 2002 .

[165]  Dnyanesh G. Rajpathak,et al.  An Ontological formalization of the planning task Conference or Workshop Item , 2004 .

[166]  Rupert Brown,et al.  Group Processes: Dynamics Within and Between Groups , 1988 .

[167]  Jakob E. Bardram Plans as Situated Action: An Activity Theory Approach to Workflow Systems , 1997 .

[168]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[169]  Sidney W. A. Dekker,et al.  To Intervene or not to Intervene: The Dilemma of Management by Exception , 1999, Cognition, Technology & Work.

[170]  Sang M. Lee,et al.  An exploratory cognitive DSS for strategic decision making , 2003, Decis. Support Syst..

[171]  Henrik Eriksson,et al.  Text-based Analysis for Command and Control Researchers : The Workflow Visualizer Approach , 2011 .

[172]  Martin van Creveld,et al.  The Art of War : War and Military Thought , 2000 .

[173]  Jiri Trnka,et al.  Role-Playing Exercise - A Real-Time Approach to Study Collaborative Command and Control , 2006 .

[174]  Dominique L. Scapin,et al.  What do people recall about their documents?: implications for desktop search tools , 2007, IUI '07.

[175]  Marcus Herzog,et al.  Using Ontologies for Extracting Product Features from Web Pages , 2006, SEMWEB.

[176]  Johan Jenvald,et al.  Critique and Visualization as decision support for mass-casualty emergency management , 2005 .

[177]  Jihie Kim,et al.  Supporting plan authoring and analysis , 2003, IUI '03.

[178]  J. Novak The Theory Underlying Concept Maps and How To Construct Them , 2004 .

[179]  Henrik Eriksson,et al.  Message classification as a basis for studying command and control communications—an evaluation of machine learning approaches , 2011, Journal of Intelligent Information Systems.

[180]  Alexander Kalloniatis,et al.  Formalization and Agility in Military Headquarters Planning , 2010 .

[181]  Steve Russ,et al.  A new paradigm for computer-based decision support , 2002, Decis. Support Syst..

[182]  Bijan Fazlollahi,et al.  The Effectiveness of Decisional Guidance: An Empirical Evaluation , 2001, Decis. Sci..

[183]  Graham Pervan,et al.  A critical analysis of decision support systems research , 2005, J. Inf. Technol..

[184]  P. Feyerabend,et al.  Against Method: Outline of an Anarchistic Theory of Knowledge , 1976 .

[185]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[186]  Janice Redish,et al.  User and task analysis for interface design , 1998 .

[187]  Kenneth D. Forbus,et al.  Sketching for military courses of action diagrams , 2003, IUI '03.

[188]  Henrik Eriksson,et al.  Analysis tools in the study of distributed decision-making: a meta-study of command and control research , 2012, Cognition, Technology & Work.

[189]  Bowen Hui,et al.  What is Initiative? , 1998, User Modeling and User-Adapted Interaction.

[190]  James F. Allen,et al.  Human-Machine Collaborative Planning , 2002 .

[191]  David Mendonça,et al.  A Cognitive Model of Improvisation in Emergency Management , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[192]  Northrup Fowler,et al.  The ARPA-Rome Knowledge-Based Planning and Scheduling Initiative , 1995, IEEE Expert.

[193]  Frederico T. Fonseca,et al.  Toward an alternative notion of information systems ontologies: Information engineering as a hermeneutic enterprise , 2005, J. Assoc. Inf. Sci. Technol..

[194]  Arthur K. Cebrowski,et al.  Network-Centric Warfare: Its Origin and Future , 1998 .

[195]  Peter van Beek,et al.  FROM PLAN CRITIQUING TO CLARIFICATION DIALOGUE FOR COOPERATIVE RESPONSE GENERATION * , 1993, Comput. Intell..

[196]  John Levine,et al.  O-P�: Supporting the Planning Process using Open Planning Process Panels , 2003, IEEE Intell. Syst..

[197]  Lin Zhang,et al.  Decision-Theoretic Military Operations Planning , 2004, ICAPS.

[198]  Preben Mogensen,et al.  Towards a Provotyping Approach in Systems Development , 1992, Scand. J. Inf. Syst..

[199]  Jayant Madhavan,et al.  Personal information management with SEMEX , 2005, SIGMOD '05.

[200]  Terry Winograd,et al.  Understanding computers and cognition - a new foundation for design , 1987 .

[201]  William J. Clancey,et al.  NEOMYCIN: Reconfiguring a Rule-Based Expert System for Application to Teaching , 1981, IJCAI.

[202]  Ian H. Witten,et al.  Data mining - practical machine learning tools and techniques, Second Edition , 2005, The Morgan Kaufmann series in data management systems.

[203]  Erik Hollnagel,et al.  Principles for modelling function allocation , 2000, Int. J. Hum. Comput. Stud..

[204]  Björn Johansson,et al.  Developing critiquing systems for network organizations. , 2004 .

[205]  Ask Not What's Inside Your Head, but What Your Head's Inside of , 1994 .

[206]  Magnus Sahlgren,et al.  Terminology mining in social media , 2009, CIKM.

[207]  Barbara Hayes-Roth,et al.  A Cognitive Model of Planning , 1979, Cogn. Sci..

[208]  M. Argyle The social psychology of work , 1972 .

[209]  Ulf Hamberg NBF- Förmågan att se på andra sidan kullen eller ”Kejsarens nya kläder" , 2010 .

[210]  Chung Hee Hwang,et al.  The TRAINS project: a case study in building a conversational planning agent , 1994, J. Exp. Theor. Artif. Intell..

[211]  Susan T. Dumais,et al.  A Bayesian Approach to Filtering Junk E-Mail , 1998, AAAI 1998.

[212]  Michael J. Cafarella,et al.  Ontology-Driven Information Extraction with OntoSyphon , 2006, SEMWEB.

[213]  H. Rittel,et al.  Dilemmas in a general theory of planning , 1973 .

[214]  Peter Thunholm,et al.  Military decision making and planning : towards a new prescriptive model , 2003 .

[215]  Mark Weiser,et al.  The world is not a desktop , 1994, INTR.

[216]  Stephen F. Smith,et al.  Configurable, Mixed-Initiative Systems for Planning and Scheduling , 1996 .

[217]  J. Berger,et al.  A Survey of Military Planning Systems , 2004 .

[218]  Shu-hsien Liao,et al.  Case-based decision support system: Architecture for simulating military command and control , 2000, Eur. J. Oper. Res..

[219]  Mabry Tyson,et al.  A Mixed-initiative Framework for Robust Plan Sketching , 2003, ICAPS.

[220]  Leo Sauermann,et al.  Semantic Desktop 2.0: The Gnowsis Experience , 2006, International Semantic Web Conference.

[221]  Siegfried Handschuh,et al.  SALT - Semantically Annotated LaTeX for scientific publications , 2007 .

[222]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[223]  Olle Bälter,et al.  Bifrost inbox organizer: giving users control over the inbox , 2002, NordiCHI '02.

[224]  Pär-Anders Albinsson,et al.  Communication Visualization – An Aid to Military Command and Control Evaluation , 2001 .

[225]  Henrik Eriksson The semantic-document approach to combining documents and ontologies , 2007, Int. J. Hum. Comput. Stud..

[226]  Allen W. Batschelet Effects-Based Operations: A New Operational Model? , 2002 .

[227]  Xin Wang,et al.  Discovering Structured Event Logs from Unstructured Audit Trails for Workflow Mining , 2009, ISMIS.

[228]  Sidney W. A. Dekker,et al.  MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination , 2002, Cognition, Technology & Work.

[229]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[230]  Stephen Potter,et al.  Using I-X Process Panels as Intelligent To-Do Lists for Agent Coordination in Emergency Response , 2006 .

[231]  Daniel Gonzales Evolution of the Air Campaign Planning Process and the Contingency Theater Automated Planning System (Ctaps) , 1996 .

[232]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[233]  Tessa A. Lau,et al.  Automated email activity management: an unsupervised learning approach , 2005, IUI.

[234]  Austin Tate,et al.  Co-OPR: Design and Evaluation of Collaborative Sensemaking and Planning Tools for Personnel Recovery , 2006 .

[235]  Pär-Anders Albinsson,et al.  Representing Military Units Using Nested Convex Hulls : Coping With Complexity in Command and Control , 2002 .

[236]  Tom Sheridan,et al.  Theater Battle Management Core System Systems Engineering Case Study , 2011 .

[237]  Rogier Woltjer On how constraints shape action , 2005 .

[238]  H. Simon Rational Decision Making in Business Organizations , 1978 .

[239]  Yarden Katz,et al.  Pellet: A practical OWL-DL reasoner , 2007, J. Web Semant..

[240]  G. Klein,et al.  Decision Making in Action: Models and Methods , 1993 .

[241]  Robert E. Kraut,et al.  Understanding email use: predicting action on a message , 2005, CHI.

[242]  Magnus Rosell Text Clustering Exploration : Swedish Text Representation and Clustering Results Unraveled , 2009 .

[243]  John M. Flach,et al.  Visualizing cognitive systems: getting past block diagrams , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[244]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[245]  Izak Benbasat,et al.  The Case Research Strategy in Studies of Information Systems , 1987, MIS Q..

[246]  Stefan Arnborg,et al.  Bayesian Games for Threat Prediction and Situation Analysis , 2004 .

[247]  Stephanie Guerlain,et al.  Designing Decision Support Systems for Revolutionary Command and Control Domains , 2003 .

[248]  Barry G. Silverman,et al.  Engineering experts critics for cooperative systems , 1993, Knowl. Eng. Rev..

[249]  David R. Traum,et al.  Negotiation over tasks in hybrid human-agent teams for simulation-based training , 2003, AAMAS '03.

[250]  Gary W. King,et al.  A Knowledge Acquisition Tool for Course of Action Analysis , 2003, IAAI.

[251]  Ilze Zigurs,et al.  A Theory of Task/Technology Fit and Group Support Systems Effectiveness , 1998, MIS Q..