Calibrating information users’ views on relevance: A social representations approach

The purpose of this study is to investigate how information users view the concept of relevance and make their judgement(s) on relevant information through the framework of social representations theory. More specifically, this study attempts to address the questions of what users view as the constituent concepts of relevance, what are core and peripheral concepts of relevance, and how these concepts are structured by applying a structural analysis approach of social representations theory. We employ a free word association method for data collection. Two hundred and forty four information users of public and academic libraries responded to questionnaires on their relevance judgement criteria. Collected data were content analysed and assessed using weighted frequency, similarity measure, and core/periphery measurements to identify key elements of relevance and to differentiate core and periphery elements of relevance. Results show that four out of 26 emerged elements (concepts) are core and 22 are periphery elements of the concept of relevance. The findings of this study provide a quantitative measure of weighing various elements of relevance and the internal structure of the concept of relevance from users’ perspectives providing enhancements for search algorithms with quantitative metadata support.

[1]  Norbert Schmitt,et al.  Quantifying word association responses: what is native-like? , 1998 .

[2]  Robert N. Oddy,et al.  INFORMATION RETRIEVAL THROUGH MAN‐MACHINE DIALOGUE , 1977 .

[3]  E. Durkheim FROM THE ELEMENTARY FORMS OF THE RELIGIOUS LIFE , 1996, The New Economic Sociology.

[4]  Jean-Claude Abric,et al.  Central System, Peripheral System: Their Functions and Roles in the Dynamics of Social Representations , 1993 .

[5]  Myke Gluck,et al.  Exploring the Relationship between User Satisfaction and Relevance in Information Systems , 1996, Inf. Process. Manag..

[6]  Michael Eisenberg,et al.  Order effects: A study of the possible influence of presentation order on user judgments of document relevance , 1988, J. Am. Soc. Inf. Sci..

[7]  Michael B. Eisenberg,et al.  A re-examination of relevance: toward a dynamic, situational definition , 1990, Inf. Process. Manag..

[8]  Peter G. B. Enser Image Retrieval: Theory and Research , 2004, J. Documentation.

[9]  Mark Sanderson,et al.  A review of factors influencing user satisfaction in information retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[10]  Joao Wachelke,et al.  CHANGING MASKS: A MASKING EFFECT ON YOUNG PEOPLE'S SOCIAL REPRESENTATION ON AGING? , 2008 .

[11]  T. Saracevic,et al.  Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance , 2007, J. Assoc. Inf. Sci. Technol..

[12]  Carol L. Barry User-Defined Relevance Criteria: An Exploratory Study , 1994, J. Am. Soc. Inf. Sci..

[13]  Davide Nicolini Comparing Methods for Mapping Organizational Cognition , 1999 .

[14]  Wolfgang Wagner,et al.  Theory and method of social representations , 1999 .

[15]  Carlos José Parales Quenza On the Structural Approach to Social Representations , 2005 .

[16]  Sonja Wiley-Patton,et al.  Conducting Social Cognition Research in IS: A Methodology for Eliciting and Analyzing Social Representations , 2009, Commun. Assoc. Inf. Syst..

[17]  K. Deaux,et al.  Representations of the social: Bridging theoretical traditions. , 2001 .

[18]  José Luis Vicedo González,et al.  TREC: Experiment and evaluation in information retrieval , 2007, J. Assoc. Inf. Sci. Technol..

[19]  R. Weber Basic Content Analysis , 1986 .

[20]  Michael B. Eisenberg Measuring relevance judgments , 1988, Inf. Process. Manag..

[21]  A. Eagly,et al.  Attitudes and opinions. , 1978, Annual review of psychology.

[22]  Tefko Saracevic Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance , 2007 .

[23]  William E. Arnold,et al.  The Likert‐type scale , 1967 .

[24]  William R. Hersh,et al.  An Evaluation of Interactive Boolean and Natural Language Searching with an Online Medical Textbook , 1995, J. Am. Soc. Inf. Sci..

[25]  C. W. Cleverdon,et al.  The ASLIB CRANFIELD RESEARCH PROJECT ON The COMPARATIVE EFFICIENCY OF INDEXING SYSTEMS , 1960 .

[26]  B. Dervin,et al.  Information needs and uses. , 1986 .

[27]  Linda Schamber,et al.  User Criteria in Relevance Evaluation: Toward Development of a Measurement Scale. , 1996 .

[28]  Martin Dillon,et al.  FASIT: A fully automatic syntactically based indexing system , 1983, J. Am. Soc. Inf. Sci..

[29]  C. Herzlich Health and illness;: A social psychological analysis , 1973 .

[30]  Ellen M. Voorhees,et al.  TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing) , 2005 .

[31]  Diane H. Sonnenwald,et al.  User perspectives on relevance criteria: A comparison among relevant, partially relevant, and not-relevant judgments , 2002, J. Assoc. Inf. Sci. Technol..

[32]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[33]  W. Wells Defining relevance , 2000, Genome Biology.

[34]  Suzanne D. Pawlowski,et al.  Focusing the research agenda on burnout in IT: social representations of burnout in the profession , 2007, Eur. J. Inf. Syst..

[35]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[36]  Martin G. Everett,et al.  Models of core/periphery structures , 2000, Soc. Networks.

[37]  C. Flament,et al.  C ONSENSUS , SALIENCE AND NECESSITY IN SOCIAL REPRESENTATIONS - TECHNICAL NOTE * , 1994 .

[38]  J Rierdan Word associations of socially isolated adolescents. , 1980, Journal of abnormal psychology.

[39]  Tasos Hovardas,et al.  Word Associations as a Tool for Assessing Conceptual Change in Science Education. , 2006 .

[40]  Pascal Moliner A two‐dimensional model of social representations , 1995 .

[41]  Judith A. Tessier New Measures of User Satisfaction With Computer Based Literature Searches. , 1977 .

[42]  W. Wagner,et al.  Relevance, discourse and the ‘hot’ stable core social representations—A structural analysis of word associations , 1996 .

[43]  Chengliang Liu,et al.  The dynamics of interactive information retrieval, Part II: An empirical study from the activity theory perspective , 2007, J. Assoc. Inf. Sci. Technol..

[44]  Mark Sanderson,et al.  A review of factors influencing user satisfaction in information retrieval , 2010 .