When complexity becomes interesting

How to provide users a positive experience during interaction with information (i.e., the “Information eXperience” (IX)) is still an open question. As a starting point, this work investigates how the emotion of interest can be influenced by modifying the complexity of the information presented to users. The appraisal theory of interest suggests a “sweet spot” where interest will be at its peak: information that is novel and complex yet still comprehensible. This “sweet spot” is approximated using two studies. Study One develops a computational model of textual complexity founded on psycholinguistic theory on processing difficulty. The model was trained and tested on 12,420 articles, achieving a classification performance of 90.87% on two classes of complexity. Study Two puts the model to its ultimate test: Its application to change the user's IX. Using 18 news articles the influence of complexity on interest and its appraisals is unveiled. A structural equation model shows a positive influence of complexity on interest, yet a negative influence of comprehensibility, confirming a seemingly paradoxical relationship between complexity and interest. By showing when complexity becomes interesting, this paper shows how information systems can use the model of textual complexity to construct an interesting IX.

[1]  Anton Nijholt,et al.  A user experience model for tangible interfaces for children , 2015, Int. J. Arts Technol..

[2]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[3]  D. Berlyne.  Conflict, arousal, and curiosity , 2014 .

[4]  Lucas Bueno,et al.  More of the same , 2014, Antiquity.

[5]  caroline sevennec,et al.  Read the news , 2014 .

[6]  A. Maslow.  A Theory of Human Motivation , 1943 .

[7]  Egon L. van den Broek,et al.  Making the news interesting: understanding the relationship between familiarity and interest , 2012, IIiX.

[8]  Mounia Lalmas,et al.  Models of user engagement , 2012, UMAP.

[9]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[10]  Rebekah George Benjamin.  Reconstructing Readability: Recent Developments and Recommendations in the Analysis of Text Difficulty , 2012 .

[11]  Lisa Linnenbrink-Garcia,et al.  Academic Emotions and Student Engagement , 2012 .

[12]  Yashar Moshfeghi,et al.  Role of emotion in information retrieval , 2012 .

[13]  Bart P. Knijnenburg,et al.  Explaining the user experience of recommender systems , 2012, User Modeling and User-Adapted Interaction.

[14]  John Riedl,et al.  Recommender systems: from algorithms to user experience , 2012, User Modeling and User-Adapted Interaction.

[15]  Ryen W. White,et al.  Personalizing web search results by reading level , 2011, CIKM '11.

[16]  Tiffany A. Whittaker.  A Beginner's Guide to Structural Equation Modeling (3rd ed.) , 2011 .

[17]  Daniel A. Connelly.  Applying Silvia's Model of Interest to Academic Text: Is There a Third Appraisal?. , 2011 .

[18]  Heeyoung Lee,et al.  Stanford’s Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task , 2011, CoNLL Shared Task.

[19]  Anton Nijholt,et al.  Designing a Museum Multi-touch Table for Children , 2011, INTETAIN.

[20]  T Florian Jaeger,et al.  On language 'utility': processing complexity and communicative efficiency. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[21]  Erez Lieberman Aiden,et al.  Quantitative Analysis of Culture Using Millions of Digitized Books , 2010, Science.

[22]  Yunhyong Kim,et al.  Why did you pick that? Visualising relevance criteria in exploratory search , 2010, International Journal on Digital Libraries.

[23]  Leif Azzopardi,et al.  Finding interest in the stream , 2011, ASIST.

[24]  Erik D. Reichle,et al.  Models of the reading process. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[25]  Lijun Feng,et al.  A Comparison of Features for Automatic Readability Assessment , 2010, COLING.

[26]  Egon L. van den Broek,et al.  Using complexity measures in information retrieval , 2010, IIiX.

[27]  Frans van der Sluis,et al.  A Closer Look at Children's Information Retrieval Usage , 2010 .

[28]  Egon L. van den Broek,et al.  Aiming for user experience in information retrieval: towards user-centered relevance (UCR) , 2010, SIGIR.

[29]  Monica Landoni,et al.  Interactive Search Interfaces for Young Children - The PuppyIR Approach , 2010, 2010 10th IEEE International Conference on Advanced Learning Technologies.

[30]  Leanne Bowler,et al.  The self-regulation of curiosity and interest during the information search process of adolescent students , 2010 .

[31]  Arthur C. Graesser,et al.  Coh-Metrix: Capturing Linguistic Features of Cohesion , 2010 .

[32]  Daniel Jurafsky,et al.  Parsing to Stanford Dependencies: Trade-offs between Speed and Accuracy , 2010, LREC.

[33]  Dawn Iacobucci,et al.  Structural Equations Modeling: Fit Indices, Sample Size, and Advanced Topics , 2010 .

[34]  Jacek Gwizdka,et al.  The role of subjective factors in the information search process , 2009, J. Assoc. Inf. Sci. Technol..

[35]  Ioannis Konstas,et al.  Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance , 2009, ACM Multimedia.

[36]  Sean Bechhofer,et al.  Visual complexity and aesthetic perception of web pages , 2008, SIGDOC '08.

[37]  D. McNamara,et al.  Assessing Text Readability Using Cognitively Based Indices , 2008 .

[38]  Joemon M. Jose,et al.  Affective feedback: an investigation into the role of emotions in the information seeking process , 2008, SIGIR '08.

[39]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[40]  Nicholas J. Belkin,et al.  Some(what) grand challenges for information retrieval , 2008, SIGF.

[41]  P. Silvia.  Interest—The Curious Emotion , 2008 .

[42]  Paul J. Silvia,et al.  Appraisal components and emotion traits: Examining the appraisal basis of trait curiosity , 2008 .

[43]  Sven Hartrumpf,et al.  A Readability Checker with Supervised Learning Using Deep Indicators , 2008, Informatica.

[44]  D. Powers.  Evaluation: From Precision, Recall and F-Factor to ROC, Informedness, Markedness & Correlation , 2008 .

[45]  T. Asano,et al.  ENTROPY , RELATIVE ENTROPY , AND MUTUAL INFORMATION , 2008 .

[46]  E. Thorndike.  The Teacher's Word Book , 2007 .

[47]  SaracevicTefko.  Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance , 2007 .

[48]  Mark Baillie,et al.  The relative effects of knowledge, interest and confidence in assessing relevance , 2007, J. Documentation.

[49]  Scott B. Huffman,et al.  How well does result relevance predict session satisfaction? , 2007, SIGIR.

[50]  William H. DuBay.  The Classic Readability Studies. , 2007 .

[51]  William Stafford Noble,et al.  Support vector machine , 2013 .

[52]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[53]  P. Gordon,et al.  Reading words in discourse: the modulation of lexical priming effects by message-level context. , 2006, Behavioral and cognitive neuroscience reviews.

[54]  Chantel S Prat,et al.  Assessing text representations with recognition: The interaction of domain knowledge and text coherence. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[55]  J. Brockner,et al.  Value from hedonic experience and engagement. , 2006, Psychological review.

[56]  S. Hidi,et al.  The Four-Phase Model of Interest Development , 2006 .

[57]  Yunjie Calvin Xu,et al.  Relevance judgment: What do information users consider beyond topicality? , 2006, J. Assoc. Inf. Sci. Technol..

[58]  P. Silvia.  Exploring the Psychology of Interest , 2006 .

[59]  Stefano Mizzaro,et al.  A Classification of IR Effectiveness Metrics , 2006, ECIR.

[60]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[61]  Clinton L. Johns,et al.  Chapter 20 – Comprehension Ability in Mature Readers , 2006 .

[62]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[63]  Dianne M. Finkelstein,et al.  A Beginner's Guide to Structural Equation Modeling , 2005, Technometrics.

[64]  Kevyn Collins-Thompson,et al.  Predicting reading difficulty with statistical language models , 2005, J. Assoc. Inf. Sci. Technol..

[65]  Donna K. Harman,et al.  Novelty Detection: The TREC Experience , 2005, HLT.

[66]  Grant T. Harris,et al.  Comparing Effect Sizes in Follow-Up Studies: ROC Area, Cohen's d, and r , 2005, Law and human behavior.

[67]  Mirella Lapata,et al.  Automatic Evaluation of Text Coherence: Models and Representations , 2005, IJCAI.

[68]  P. Silvia.  What is interesting? Exploring the appraisal structure of interest. , 2005, Emotion.

[69]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[70]  Dobb's journal staff.  Of interest , 2001, Administration in mental health.

[71]  Ian Ruthven,et al.  Integrating approaches to relevance , 2005 .

[72]  Ping Zhang,et al.  Flow in Computer-Mediated Environments: Promises and Challenges , 2005, Commun. Assoc. Inf. Syst..

[73]  Otis Gospodnetic,et al.  Lucene in Action , 2004 .

[74]  William H. DuBay.  The Principles of Readability. , 2004 .

[75]  Michael J Cortese,et al.  Visual word recognition of single-syllable words. , 2004, Journal of experimental psychology. General.

[76]  Michael A Babyak,et al.  What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models , 2004, Psychosomatic medicine.

[77]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[78]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[79]  Peretz Shoval,et al.  Information Filtering: Overview of Issues, Research and Systems , 2001, User Modeling and User-Adapted Interaction.

[80]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[81]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[82]  Kurt Hornik,et al.  The support vector machine under test , 2003, Neurocomputing.

[83]  Pia Borlund,et al.  The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[84]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[85]  R. Shillcock,et al.  Low-level predictive inference in reading: the influence of transitional probabilities on eye movements , 2003, Vision Research.

[86]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[87]  Dale Schuurmans,et al.  Combining Naive Bayes and n-Gram Language Models for Text Classification , 2003, ECIR.

[88]  John Hale,et al.  The Information Conveyed by Words in Sentences , 2003, Journal of psycholinguistic research.

[89]  K. Scherer,et al.  Appraisal processes in emotion. , 2003 .

[90]  Pia Borlund,et al.  The IIR evaluation model: a framework for evaluation of interactive information retrieval systems , 2003, Inf. Res..

[91]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[92]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[93]  Ephraim R. McLean,et al.  Information systems success revisited , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[94]  E. Fry,et al.  Readability versus Leveling. , 2002 .

[95]  Y. Rogers,et al.  Interaction Design , 2002 .

[96]  Ellen M. Voorhees,et al.  The Philosophy of Information Retrieval Evaluation , 2001, CLEF.

[97]  Mark Sadoski,et al.  Resolving the Effects of Concreteness on Interest, Comprehension, and Learning Important Ideas From Text , 2001 .

[98]  P. Silvia.  Interest and Interests: The Psychology of Constructive Capriciousness , 2001 .

[99]  Gregory Schraw,et al.  Situational Interest: A Review of the Literature and Directions for Future Research , 2001 .

[100]  Amanda Spink,et al.  Regions and levels: Measuring and mapping users' relevance judgments , 2001, J. Assoc. Inf. Sci. Technol..

[101]  Young-Gul Kim,et al.  Extending the TAM for a World-Wide-Web context , 2000, Inf. Manag..

[102]  Martin Porter,et al.  Snowball: A language for stemming algorithms , 2001 .

[103]  David H. Jonassen,et al.  Toward a design theory of problem solving , 2000 .

[104]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[105]  E. Gibson.  The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[106]  G. Zipf.  The Psycho-Biology Of Language: AN INTRODUCTION TO DYNAMIC PHILOLOGY , 1999 .

[107]  Yves Bestgen,et al.  The impact of connectives and anaphoric expressions on expository discourse comprehension , 1999 .

[108]  E. Gibson.  Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[109]  Peter Ingwersen,et al.  Measures of relative relevance and ranked half-life: performance indicators for interactive IR , 1998, SIGIR '98.

[110]  P. Jordan.  Human factors for pleasure in product use. , 1998, Applied ergonomics.

[111]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[112]  Erik D. Reichle,et al.  Toward a model of eye movement control in reading. , 1998, Psychological review.

[113]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[114]  Gregory Schraw,et al.  Situational Interest in Literary Text , 1997, Contemporary educational psychology.

[115]  清川 英男,et al.  CHALL, J. S. and DALE, E. (1995) Readability Revisited : The New Dale-Chall Readability Formula., Brookline Books , 1996 .

[116]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[117]  Susan Dunman,et al.  Seeking meaning: A process approach to library and information services , 1996 .

[118]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[119]  Ulrich Schiefele,et al.  Topic interest, text representation, and quality of experience , 1996 .

[120]  Ulrich Schiefele,et al.  Topic interest and free recall of expository text. , 1996 .

[121]  Myke Gluck,et al.  Exploring the Relationship between User Satisfaction and Relevance in Information Systems , 1996, Inf. Process. Manag..

[122]  G. Clore,et al.  Feelings and phenomenal experiences , 1996 .

[123]  W. Kintsch,et al.  Are Good Texts Always Better? Interactions of Text Coherence, Background Knowledge, and Levels of Understanding in Learning From Text , 1996 .

[124]  Gregory Schraw,et al.  Cognitive processes in well‐defined and ill‐defined problem solving , 1995 .

[125]  Mark Rosenstein,et al.  Recommending and evaluating choices in a virtual community of use , 1995, CHI '95.

[126]  J. Chall,et al.  Readability revisited : the new Dale-Chall readability formula , 1995 .

[127]  Trudi E. Jacobson,et al.  Seeking meaning: A process approach to library and information services: by Carol Collier Kuhlthau. Norwood, NJ: Ablex, 1993. 219p. $45.00 (cloth). ISBN 0-89391-968-3. $24.50 (paper). ISBN 1-56750-019-6. LC 92-40770 , 1994 .

[128]  Louise T. Su.  The Relevance of Recall and Precision in User Evaluation , 1994, J. Am. Soc. Inf. Sci..

[129]  Carol L. Barry.  User-Defined Relevance Criteria: An Exploratory Study , 1994, J. Am. Soc. Inf. Sci..

[130]  Linda Schamber.  Relevance and Information Behavior. , 1994 .

[131]  David A. Balota,et al.  Visual Word Recognition: The Journey from Features to Meaning (A Travel Update) , 2006 .

[132]  W. Bruce Croft,et al.  Information filtering and information retrieval: two sides of the same coin? , 1992, CACM.

[133]  J. G. Carlson,et al.  Psychology of Emotion , 1991 .

[134]  R. Lazarus.  Progress on a cognitive-motivational-relational theory of emotion. , 1991, The American psychologist.

[135]  Graeme Hirst,et al.  Lexical Cohesion Computed by Thesaural relations as an indicator of the structure of text , 1991, CL.

[136]  Suzanne Hidi,et al.  Interest and Its Contribution as a Mental Resource for Learning , 1990 .

[137]  J. Reeve,et al.  The interest-enjoyment distinction in intrinsic motivation , 1989 .

[138]  M. Csíkszentmihályi,et al.  Optimal experience in work and leisure. , 1989, Journal of personality and social psychology.

[139]  R. Descartes,et al.  The passions of the soul , 1989 .

[140]  C. Perfetti.  Verbal efficiency in reading ability. , 1988 .

[141]  K. Rayner,et al.  Parafoveal word processing during eye fixations in reading: Effects of word frequency , 1986, Perception & psychophysics.

[142]  D. Sperber,et al.  Relevance: Communication and Cognition , 1989 .

[143]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[144]  T. Saracevic,et al.  Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance , 2007, J. Assoc. Inf. Sci. Technol..

[145]  Tefko Saracevic,et al.  RELEVANCE: A review of and a framework for the thinking on the notion in information science , 1997, J. Am. Soc. Inf. Sci..

[146]  D. Berlyne.  Behaviourism? Cognitive theory? Humanistic psychology? To Hull with them all. , 1975 .

[147]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[148]  D. Berlyne.  Aesthetics and psychobiology , 1975 .

[149]  R. Schvaneveldt,et al.  Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. , 1971, Journal of experimental psychology.

[150]  D. Berlyne.  Novelty, complexity, and hedonic value , 1970 .

[151]  H. Day.  Evaluations of subjective complexity, pleasingness and interestingness for a series of random polygons varying in complexity , 1967 .

[152]  N H Frijda,et al.  Problems of computer simulation. , 1967, Behavioral science.

[153]  Harry Munsinger,et al.  Uncertainty, structure, and preference. , 1964 .

[154]  E. Hess,et al.  Pupil Size as Related to Interest Value of Visual Stimuli , 1960, Science.

[155]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[156]  E. McGinnies,et al.  Visual-recognition thresholds as a function of word length and word frequency. , 1952, Journal of experimental psychology.

[157]  R. Flesch.  A new readability yardstick. , 1948, The Journal of applied psychology.

[158]  C. L. Hull.  Principles of behavior : an introduction to behavior theory , 1943 .