CoNotate: Suggesting Queries Based on Notes Promotes Knowledge Discovery

When exploring a new domain through web search, people often struggle to articulate queries because they lack domain-specific language and well-defined informational goals. Perhaps search tools rely too much on the query to understand what a searcher wants. Towards expanding this contextual understanding of a user during exploratory search, we introduce a novel system, CoNotate, which offers query suggestions based on analyzing the searcher’s notes and previous searches for patterns and gaps in information. To evaluate this approach, we conducted a within-subjects study where participants (n=38) conducted exploratory searches using a baseline system (standard web search) and the CoNotate system. The CoNotate approach helped searchers issue significantly more queries, and discover more terminology than standard web search. This work demonstrates how search can leverage user-generated content to help people get started when exploring complex, multi-faceted information spaces.

[1]  Xue Dong Yang,et al.  A Comparative User Study of Web Search Interfaces: HotMap, Concept Highlighter, and Google , 2006, 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2006 Main Conference Proceedings)(WI'06).

[2]  Pia Borlund,et al.  The IIR evaluation model: a framework for evaluation of interactive information retrieval systems , 2003, Inf. Res..

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  M. J. Wilkenfeld,et al.  Similarity and emergence in conceptual combination , 2001 .

[5]  Pertti Vakkari,et al.  Changes in Search Tactics and Relevance Judgements when Preparing a Research Proposal A Summary of the Findings of a Longitudinal Study , 2001, Information Retrieval.

[6]  Michael S. Bernstein,et al.  Information ! Scraps : ! ! How ! and ! Why ! Information ! Eludes ! our ! Personal ! Information ! Management , 2022 .

[7]  Jaime Teevan,et al.  Implicit feedback for inferring user preference: a bibliography , 2003, SIGF.

[8]  Satoshi Shimada,et al.  Can Disputed Topic Suggestion Enhance User Consideration of Information Credibility in Web Search? , 2016, HT.

[9]  Tovi Grossman,et al.  Ambient help , 2011, CHI.

[10]  Meredith Ringel Morris,et al.  A survey of collaborative web search practices , 2008, CHI.

[11]  Karl Gyllstrom,et al.  A comparison of query and term suggestion features for interactive searching , 2009, SIGIR.

[12]  Karl Gyllstrom,et al.  Effects of popularity and quality on the usage of query suggestions during information search , 2010, CHI.

[13]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[14]  F. D. Kahn,et al.  A survey of note-taking practices , 1992 .

[15]  Bernard J. Jansen,et al.  Evaluating the effectiveness of and patterns of interactions with automated searching assistance , 2005, J. Assoc. Inf. Sci. Technol..

[16]  Catherine C. Marshall,et al.  Saving and using encountered information: implications for electronic periodicals , 2005, CHI.

[17]  Aniket Kittur,et al.  SearchLens: composing and capturing complex user interests for exploratory search , 2019, IUI.

[18]  W. Bruce Croft,et al.  Asking Clarifying Questions in Open-Domain Information-Seeking Conversations , 2019, SIGIR.

[19]  Kevyn Collins-Thompson,et al.  Towards searching as a learning process: A review of current perspectives and future directions , 2016, J. Inf. Sci..

[20]  Susan T. Dumais,et al.  Bringing order to the Web: automatically categorizing search results , 2000, CHI.

[21]  Zhiyuan Liu,et al.  Query Suggestion with Feedback Memory Network , 2018, WWW.

[22]  Wiebke Wagner,et al.  Steven Bird, Ewan Klein and Edward Loper: Natural Language Processing with Python, Analyzing Text with the Natural Language Toolkit , 2010, Lang. Resour. Evaluation.

[23]  Scott R. Klemmer,et al.  Example-centric programming: integrating web search into the development environment , 2010, CHI.

[24]  Matthew Banta,et al.  What do exploratory searchers look at in a faceted search interface? , 2009, JCDL '09.

[25]  Ben Carterette,et al.  Overview of the TREC 2014 Session Track , 2014, TREC.

[26]  Itiel E. Dror,et al.  Optimising the use of note-taking as an external cognitive aid for increasing learning , 2009, Br. J. Educ. Technol..

[27]  Robert G. Capra,et al.  Towards Better Support for Exploratory Search through an Investigation of Notes-to-self and Notes-to-share , 2019, SIGIR.

[28]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[29]  Susan R. Fussell,et al.  Effects of visualization and note-taking on sensemaking and analysis , 2013, CHI.

[30]  L. R. Rasmussen,et al.  In information retrieval: data structures and algorithms , 1992 .

[31]  Aniket Kittur,et al.  Standing on the schemas of giants: socially augmented information foraging , 2014, CSCW.

[32]  Gilles Fauconnier,et al.  Conceptual Integration Networks , 1998, Cogn. Sci..

[33]  Bryce Allen,et al.  Cognitive and task influences on Web searching behavior , 2002, J. Assoc. Inf. Sci. Technol..

[34]  David R. Karger,et al.  Scatter/Gather: A Cluster-based Approach to Browsing Large Document Collections , 2017, SIGF.

[35]  Gary Marchionini,et al.  Examining the effectiveness of real-time query expansion , 2007, Inf. Process. Manag..

[36]  Francesco Bonchi,et al.  Do you want to take notes?: identifying research missions in Yahoo! search pad , 2010, WWW '10.

[37]  Filip Radlinski,et al.  Inferring query intent from reformulations and clicks , 2010, WWW '10.

[38]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[39]  Michael S. Bernstein,et al.  Note to self: examining personal information keeping in a lightweight note-taking tool , 2009, CHI.

[40]  David Kirsh,et al.  Thinking with external representations , 2010, AI & SOCIETY.

[41]  Ricardo A. Baeza-Yates,et al.  Query Recommendation Using Query Logs in Search Engines , 2004, EDBT Workshops.

[42]  A. Azzouz.  2011 , 2020, City.

[43]  Diane Kelly,et al.  The use of query suggestions during information search , 2014, Inf. Process. Manag..

[44]  Yusuke Yamamoto,et al.  Disputed Sentence Suggestion towards Credibility-Oriented Web Search , 2012, APWeb.

[45]  Desney S. Tan,et al.  InkSeine: In Situ search for active note taking , 2007, CHI.

[46]  Daniel Lakens,et al.  An integrative review of the cognitive costs and benefits of note-taking , 2017 .

[47]  Benjamin Rey,et al.  Generating query substitutions , 2006, WWW '06.

[48]  Eric Horvitz,et al.  SearchTogether: an interface for collaborative web search , 2007, UIST.

[49]  Max L. Wilson,et al.  A comparison of techniques for measuring sensemaking and learning within participant-generated summaries , 2013, J. Assoc. Inf. Sci. Technol..

[50]  Ryen W. White,et al.  Mining the search trails of surfing crowds: identifying relevant websites from user activity , 2008, WWW.

[51]  Jakob Grue Simonsen,et al.  A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion , 2015, CIKM.

[52]  Alessandro Micarelli,et al.  User Profiles for Personalized Information Access , 2007, The Adaptive Web.

[53]  Pertti Vakkari,et al.  Searching as learning: A systematization based on literature , 2016, J. Inf. Sci..

[54]  Fabrizio Silvestri,et al.  Mining Query Logs: Turning Search Usage Data into Knowledge , 2010, Found. Trends Inf. Retr..

[55]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[56]  Jeffrey Nichols,et al.  No Code Required: Giving Users Tools to Transform the Web , 2010 .

[57]  Ryen W. White,et al.  Studying trailfinding algorithms for enhanced web search , 2010, SIGIR.

[58]  Robert G. Capra,et al.  Tools-at-hand and learning in multi-session, collaborative search , 2010, CHI.

[59]  Chris Quintana,et al.  The Digital IdeaKeeper: Extending Digital Library Services to Scaffold Online Inquiry , 2004 .

[60]  Ricardo Baeza-Yates,et al.  Usage Data in Web Search: Benefits and Limitations , 2012, SPIRE.

[61]  Jacek Gwizdka,et al.  Searching as learning: Novel measures for information interaction research , 2014, ASIST.

[62]  Aniket Kittur,et al.  Sensemaking : Improving Sensemaking by Leveraging the Efforts of Previous Users , 2012 .

[63]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[64]  Chirag Shah,et al.  Exploring the relationships between search intentions and query reformulations , 2016, ASIST.

[65]  Venkatesh Balasubramanian,et al.  Slice: Scalable Linear Extreme Classifiers Trained on 100 Million Labels for Related Searches , 2019, WSDM.

[66]  Juho Kim,et al.  SolveDeep: A System for Supporting Subgoal Learning in Online Math Problem Solving , 2019, CHI Extended Abstracts.

[67]  Aniket Kittur,et al.  The Knowledge Accelerator: Big Picture Thinking in Small Pieces , 2016, CHI.

[68]  Gary Marchionini,et al.  Synthesis Lectures on Information Concepts, Retrieval, and Services , 2009 .

[69]  Dan Morris,et al.  SearchBar: a search-centric web history for task resumption and information re-finding , 2008, CHI.

[70]  Aniket Kittur,et al.  Unakite: Scaffolding Developers' Decision-Making Using the Web , 2019, UIST.

[71]  Chirag Shah,et al.  Searching as Learning: Exploring Search Behavior and Learning Outcomes in Learning-related Tasks , 2018, CHIIR.

[72]  Grace Hui Yang,et al.  Session Search by Direct Policy Learning , 2015, ICTIR.

[73]  RiehSoo Young,et al.  Towards searching as a learning process , 2016 .

[74]  Marcia J. Bates,et al.  Information search tactics , 1979, J. Am. Soc. Inf. Sci..

[75]  Xiaojun Yuan,et al.  Building the trail best traveled: effects of domain knowledge on web search trailblazing , 2012, CHI.

[76]  Tetsuya Sakai,et al.  Estimating Intent Types for Search Result Diversification , 2013, AIRS.

[77]  Filip Radlinski,et al.  Improving personalized web search using result diversification , 2006, SIGIR.

[78]  Robert G. Capra,et al.  Using Trails to Support Users with Tasks of Varying Scope , 2019, SIGIR.

[79]  Tetsuya Sakai,et al.  Structured query suggestion for specialization and parallel movement: effect on search behaviors , 2012, WWW.

[80]  Susan T. Dumais,et al.  Personalizing Search via Automated Analysis of Interests and Activities , 2005, SIGIR.

[81]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[82]  Ingrid Hsieh-Yee,et al.  Effects of Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers , 1993, J. Am. Soc. Inf. Sci..

[83]  Bill N. Schilit,et al.  XLibris: the active reading machine , 1998, CHI Conference Summary.

[84]  Eugene Agichtein,et al.  To hint or not: exploring the effectiveness of search hints for complex informational tasks , 2014, SIGIR.

[85]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[86]  George Veletsianos.  Cognitive and Affective Benefits of an Animated Pedagogical Agent: Considering Contextual Relevance and Aesthetics , 2007 .

[87]  Wei Huang,et al.  The exploration of objective task difficulty and domain knowledge effects on users' query formulation , 2016, ASIST.

[88]  Pernilla Qvarfordt,et al.  Leading people to longer queries , 2013, CHI.

[89]  Jaakko Peltonen,et al.  Topic-Relevance Map: Visualization for Improving Search Result Comprehension , 2017, IUI.

[90]  John T. Stasko,et al.  Be Quiet? Evaluating Proactive and Reactive User Interface Assistants , 2003, INTERACT.

[91]  Eric Horvitz,et al.  Patterns of search: analyzing and modeling Web query refinement , 1999 .

[92]  Stuart K. Card,et al.  Information foraging in information access environments , 1995, CHI '95.

[93]  Aniket Kittur,et al.  Sensemaking in a Senseless World: 2018 Workshop Abstract , 2018, CHI Extended Abstracts.

[94]  Soohyung Joo,et al.  Effects of topic familiarity and search skills on query reformulation behavior , 2013, ASIST.

[95]  Wayne G. Lutters,et al.  Understanding the micronote lifecycle: improving mobile support for informal note taking , 2004, CHI.

[96]  Skw Chu.  A novice-expert comparison in information search , 2011 .

[97]  Robert G. Capra,et al.  Differences in the Use of Search Assistance for Tasks of Varying Complexity , 2015, SIGIR.

[98]  Adam Fourney,et al.  CheatSheet: a contextual interactive memory aid for web applications , 2015, Graphics Interface.

[99]  Paul N. Bennett,et al.  Leading Conversational Search by Suggesting Useful Questions , 2020, WWW.

[100]  Yvonne Kammerer,et al.  Signpost from the masses: learning effects in an exploratory social tag search browser , 2009, CHI.

[101]  Wei-Ying Ma,et al.  Learning to cluster web search results , 2004, SIGIR '04.