Sections, categories and keywords as interest specification tools for personalised news services

Through an evaluation of system performance and user satisfaction for the Mercurio system, considers the general applicability and usefulness of different methods of specifying user interest for the general case of digital news services. Outlines the specific characteristics distinguishing such systems from more general information systems and discusses their effect. Proposes an evaluation blueprint for them starting from information retrieval procedures, existing work on search engine evaluation, and a close study of the working principles and the required evaluation according to the particular properties and conditions of the services under consideration. Presents and discusses actual evaluation results for system tests based both on real users and customised test cases. Conclusions cover the nature of the information handling tasks that digital news services are faced with, the relative merits of sections, categories and keywords with respect to this particular set of tasks, and the risks of careless application of recall and precision measures in systems such as these.

[1]  James P. Callan,et al.  Training algorithms for linear text classifiers , 1996, SIGIR '96.

[2]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[3]  Peter Willett,et al.  Estimating the recall performance of Web search engines , 1997 .

[4]  Pablo Gervás,et al.  Evaluating a User-Model Based Personalisation Architecture for Digital News Services , 2000, ECDL.

[5]  Ángeles Maldonado-Martínez,et al.  Evaluación de los principales "buscadores" desde un punto de vista documental: recogida, análisis y recuperación de recursos de información , 1998 .

[6]  Michael D. Gordon,et al.  Finding Information on the World Wide Web: The Retrieval Effectiveness of Search Engines , 1999, Inf. Process. Manag..

[7]  Xiaoying Dong,et al.  SEARCH ENGINES ON THE WORLD WIDE WEB AND INFORMATION RETRIEVAL FROM THE INTERNET: A REVIEW AND EVALUATION , 1997 .

[8]  Juan Antonio Pastor Sánchez,et al.  Un modelo para la evaluación de interfaces en sistemas de recuperación de información , 1999 .

[9]  Candy Schwartz,et al.  Web Search Engines , 1998, J. Am. Soc. Inf. Sci..

[10]  Jaideep Srivastava,et al.  First 20 precision among World Wide Web search services (search engines) , 1999 .

[11]  Fabrizio Sebastiani,et al.  A Tutorial on Automated Text Categorisation , 2000 .

[12]  Jose Maria Gomez-Hidalgo,et al.  Integrating a Lexical Database and a Training Collection for Text Categorization , 1997 .

[13]  Alberto Díaz Esteban,et al.  Nuevos sistemas de información: tendencias y evaluación , 2000 .

[14]  María-Dolores Olvera-Lobo.  Rendimiento de los sistemas de recuperación de información en la world wide web: revisión metodológica , 2000 .

[15]  María Dolores Olvera Lobo.  Rendimiento de los sistemas de recuperaciôn de información en la world wide web : Revisión metodológica , 2000 .

[16]  Umberto Straccia,et al.  User Profile Modeling and Applications to Digital Libraries , 1999, ECDL.