Improving the Usability of Hierarchical Representations for Interactively Labeling Large Image Data Sets

Image recognition systems require large image data sets for the training process. The annotation of such data sets through users requires a lot of time and effort, and thereby presents the bottleneck in the development of recognition systems. In order to simplify the creation of image recognition systems it is necessary to develop interaction concepts for optimizing the usability of labeling systems. Semi-automatic approaches are capable of solving the labeling task by clustering the image data unsupervised and presenting this ordered set to a user for manual labeling. A labeling interface based on selforganizing maps (SOM) was developed and its usability was investigated in an extensive user study with 24 participants. The evaluation showed that SOM-based visualizations are suitable for speeding up the labeling process and simplifying the task for users. Based on the results of the user study, further concepts were developed to improve the usability.

[1]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[2]  Wei-Ying Ma,et al.  Image and Video Retrieval , 2003, Lecture Notes in Computer Science.

[3]  Young-Seuk Park,et al.  Self-Organizing Map , 2008 .

[4]  Benjamin B. Bederson,et al.  PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps , 2001, UIST '01.

[5]  Helge J. Ritter,et al.  Semi-automatic acquisition and labelling of image data using SOMs , 2003, ESANN.

[6]  Benjamin Z. Yao,et al.  Introduction to a Large-Scale General Purpose Ground Truth Database: Methodology, Annotation Tool and Benchmarks , 2007, EMMCVPR.

[7]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[8]  Takayuki Itoh,et al.  CAT: A Hierarchical Image Browser Using a Rectangle Packing Technique , 2008, 2008 12th International Conference Information Visualisation.

[9]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[10]  Jane Yung-jen Hsu,et al.  KissKissBan: a competitive human computation game for image annotation , 2010, HCOMP '09.

[11]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[12]  Esa Alhoniemi,et al.  Self-organizing map in Matlab: the SOM Toolbox , 1999 .

[13]  Erkki Oja,et al.  PicSOM - content-based image retrieval with self-organizing maps , 2000, Pattern Recognit. Lett..

[14]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[15]  Jorma Laaksonen,et al.  Semantic Annotation of Image Groups with Self-organizing Maps , 2005, CIVR.

[16]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[17]  Ebroul Izquierdo,et al.  An interactive framework for image annotation through gaming , 2010, MIR '10.