How Does Domain Expertise Affect Users’ Search Interaction and Outcome in Exploratory Search?

People often conduct exploratory search to explore unfamiliar information space and learn new knowledge. While supporting the highly dynamic and interactive exploratory search is still challenging for the search system, we want to investigate which factors can make the exploratory search successful and satisfying from the user’s perspective. Previous research suggests that domain experts have different search strategies and are more successful in finding domain-specific information, but how the domain expertise level will influence users’ interaction and search outcomes in exploratory search, especially in different knowledge domains, is still unclear. In this work, via a carefully designed user study that involves 30 participants, we investigate the influence of domain expertise levels on the interaction and outcome of exploratory search in three different domains: environment, medicine, and politics. We record participants’ search behaviors, including their explicit feedback and eye fixation sequences, in a laboratory setting. With this dataset, we identify both domain-independent and domain-dependent effects on user behaviors and search outcomes. Our results extend existing research on the effect of domain expertise in search and suggest different strategies for exploiting domain expertise to support exploratory search in different knowledge domains.

[1]  Wei Huang,et al.  The exploration of objective task difficulty and domain knowledge effects on users' query formulation , 2016, ASIST.

[2]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[3]  Kevyn Collins-Thompson,et al.  Assessing Learning Outcomes in Web Search: A Comparison of Tasks and Query Strategies , 2016, CHIIR.

[4]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[5]  Yiqun Liu,et al.  When does Relevance Mean Usefulness and User Satisfaction in Web Search? , 2016, SIGIR.

[6]  Stephen J. Payne,et al.  Knowledge in the head and on the web: using topic expertise to aid search , 2008, CHI.

[7]  Sebastian Dungs,et al.  An Eye-Tracking Study of Query Reformulation , 2015, SIGIR.

[8]  Wai-Tat Fu,et al.  Exploratory information search by domain experts and novices , 2010, IUI '10.

[9]  Eric Horvitz,et al.  Patterns of search: analyzing and modeling Web query refinement , 1999 .

[10]  Suresh K. Bhavnani,et al.  Domain-specific search strategies for the effective retrieval of healthcare and shopping information , 2002, CHI Extended Abstracts.

[11]  S. Esener,et al.  Optical image inversion and edge detection based on Vertical Cavity Semiconductor Optical Amplifiers (VCSOAs) , 2005, 2005 OSA Topical Meeting on Information Photonics (IP).

[12]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[13]  Ryen W. White,et al.  Supporting Complex Search Tasks , 2014, CIKM.

[14]  Suresh K. Bhavnani,et al.  Important Cognitive Components of Domain-Specific Search Knowledge , 2001, TREC.

[15]  Ryen W. White,et al.  Lessons from the journey: a query log analysis of within-session learning , 2014, WSDM.

[16]  Barbara M. Wildemuth,et al.  The effects of domain knowledge on search tactic formulation , 2004, J. Assoc. Inf. Sci. Technol..

[17]  Ben Carterette,et al.  Overview of the TREC 2011 Session Track , 2011, TREC.

[18]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[19]  Nicholas J. Belkin,et al.  Predicting users' domain knowledge in information retrieval using multiple regression analysis of search behaviors , 2015, J. Assoc. Inf. Sci. Technol..

[20]  Xiaojun Yuan,et al.  Domain knowledge, search behaviour, and search effectiveness of engineering and science students: an exploratory study , 2005, Inf. Res..

[21]  Daqing He,et al.  Searching, browsing, and clicking in a search session: changes in user behavior by task and over time , 2014, SIGIR.

[22]  Erik D. Reichle,et al.  Direct lexical control of eye movements in reading: Evidence from a survival analysis of fixation durations , 2012, Cognitive Psychology.

[23]  Benjamin Naumann.  The Architecture Of Cognition , 2016 .

[24]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[25]  Dorota Glowacka,et al.  Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks , 2016, J. Assoc. Inf. Sci. Technol..

[26]  Yiqun Liu,et al.  Influence of Vertical Result in Web Search Examination , 2015, SIGIR.

[27]  Jacek Gwizdka,et al.  Knowledge effects on document selection in search results pages , 2011, SIGIR '11.

[28]  Ryen W. White,et al.  Struggling and Success in Web Search , 2015, CIKM.

[29]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[30]  James Allan,et al.  Extending Faceted Search to the General Web , 2014, CIKM.

[31]  Noriko Kando,et al.  Using a concept map to evaluate exploratory search , 2010, IIiX.

[32]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[33]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[34]  Dorota Glowacka,et al.  Narrow or Broad?: Estimating Subjective Specificity in Exploratory Search , 2014, CIKM.

[35]  Pertti Vakkari,et al.  Changes of search terms and tactics while writing a research proposal: A longitudinal case study , 2003, Inf. Process. Manag..

[36]  Andreas Dengel,et al.  Eye movements as implicit relevance feedback , 2008, CHI Extended Abstracts.

[37]  Yiqun Liu,et al.  Investigation of User Search Behavior While Facing Heterogeneous Search Services , 2017, WSDM.

[38]  Yiqun Liu,et al.  Different Users, Different Opinions: Predicting Search Satisfaction with Mouse Movement Information , 2015, SIGIR.

[39]  Yiqun Liu,et al.  "Satisfaction with Failure" or "Unsatisfied Success": Investigating the Relationship between Search Success and User Satisfaction , 2018, WWW.

[40]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[41]  Lynda Tamine,et al.  On the impact of domain expertise on query formulation, relevance assessment and retrieval performance in clinical settings , 2017, Inf. Process. Manag..

[42]  Jacek Gwizdka,et al.  Inferring user knowledge level from eye movement patterns , 2013, Inf. Process. Manag..