A Task-based Framework to Evaluate Evaluative Arguments

We present an evaluation framework in which the effectiveness of evaluative arguments can be measured with real users. The framework is based on the task-efficacy evaluation method. An evaluative argument is presented in the context of a decision task and measures related to its effectiveness are assessed. Within this framework, we are currently running a formal experiment to verify whether argument effectiveness can be increased by tailoring the argument to the user and by varying the degree of argument conciseness.

[1]  Alistair Knott,et al.  A data-driven methodology for motivating a set of coherence relations , 1996 .

[2]  Barbara Di Eugenio,et al.  Introduction to the Special Issue on Natural Language Generation , 1998, CL.

[3]  Michael Elhadad,et al.  Using argumentation in text generation , 1995 .

[4]  James Shaw.  Clause Aggregation Using Linguistic Knowledge , 1998, INLG.

[5]  F. H. Barron,et al.  SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement , 1994 .

[6]  Kathleen McKeown,et al.  Empirically Designing and Evaluating a New Revision-Based Model for Summary Generation , 1996, Artif. Intell..

[7]  Andrew S. Rancer,et al.  A conceptualization and measure of argumentativeness. , 1982, Journal of personality assessment.

[8]  James C. Lester,et al.  Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments , 1997, Comput. Linguistics.

[9]  R. Michael Young,et al.  Using Grice's maxim of Quantity to select the content of plan descriptions , 1999, Artif. Intell..

[10]  J. Cacioppo,et al.  Effects of need for cognition on message evaluation, recall, and persuasion. , 1983 .

[11]  Robert T. Clemen,et al.  Making Hard Decisions: An Introduction to Decision Analysis , 1997 .

[12]  Steven F. Roth,et al.  Toward an Information Visualization Workspace: Combining Multiple Means of Expression , 1997, Hum. Comput. Interact..

[13]  Giuseppe Carenini.  Evaluating Multimedia Interactive Arguments in the Context of Data Exploration Tasks , 1998 .

[14]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[15]  Johanna D. Moore,et al.  A strategy for generating evaluative arguments , 2000, INLG.

[16]  Johanna D. Moore,et al.  Does Discourse Planning Require A Special-Purpose Planner? , 1994 .

[17]  Jennifer Chu-Carroll,et al.  Collaborative Response Generation in Planning Dialogues , 1998, Comput. Linguistics.

[18]  C. F. Kao,et al.  The efficient assessment of need for cognition. , 1984, Journal of personality assessment.