Visual Saliency Based Multiple Objects Segmentation and its Parallel Implementation for Real-Time Vision Processing

This paper presents a segmentation method of multiple object regions based on visual saliency. Our method comprises three steps. First, attentional points are detected using saliency maps (SMs). Subsequently, regions of interest (RoIs) are extracted using scale-invariant feature transform (SIFT). Finally, foreground regions are extracted as object regions using GrabCut. Using RoIs as teaching signals, our method achieved automatic segmentation of multiple objects without learning in advance. As experimentally obtained results obtained using PASCAL2011 dataset, attentional points were extracted correctly from 18 images for two objects and from 25 images for single objects. We obtained segmentation accuracies: 64.1%, precision; 62.1%, recall, and 57.4%, F-measure. For real-time video image processing, we implemented our model on an IMAPCAR2 evaluation board. The processing cost was 47.5 ms for the video images of 640 × 240 pixel resolution. Moreover, we applied our method to time-series images obtained using a mobile robot. Attentional points were extracted correctly for seven images for two objects and three images for single objects from ten images. We obtained segmentation accuracies of 58.0%, precision; 63.1%, recall, and 58.1%, F-measure.

[1]  Tai Sing Lee,et al.  Image Representation Using 2D Gabor Wavelets , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Masayuki Inaba,et al.  Incremental Acquisition of Behavior Decision Model based on Interaction between Human and Robots , 2001 .

[3]  ZissermanAndrew,et al.  The Pascal Visual Object Classes Challenge , 2015 .

[4]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[5]  Pierre Marchal,et al.  A mixed digital-analog SIMD chip tailored for image perception , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[6]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[7]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[8]  Junji Yamato,et al.  Saliency-based video segmentation with graph cuts and sequentially updated priors , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[9]  Heinz Hügli,et al.  A Real Time Implementation of the Saliency-Based Model of Visual Attention on a SIMD Architecture , 2002, DAGM-Symposium.

[10]  Tetsuya TAKIGUCHI,et al.  Automatic Segmentation using Graph Cuts based on AdaBoost and Saliency Map , 2008 .

[11]  Hirokazu Madokoro,et al.  Unsupervised Indoor Scene Classification Based on Context for a Mobile Robot , 2013 .

[12]  Hirokazu Madokoro,et al.  Visual saliency based segmentation of multiple objects using variable regions of interest , 2014, 2014 14th International Conference on Control, Automation and Systems (ICCAS 2014).

[13]  Takeshi Naemura,et al.  Bounding-Box Based Segmentation with Single Min-cut Using Distant Pixel Similarity , 2010, 2010 20th International Conference on Pattern Recognition.

[14]  Tao Mei,et al.  Concurrent Multiple Instance Learning for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Yoshihiro Fujita.  Childcare Robot PaPeRo , 2006 .

[16]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Satoshi Takahashi,et al.  Parallel implementation of saliency maps for real-time robot vision , 2014, 2014 14th International Conference on Control, Automation and Systems (ICCAS 2014).

[18]  Yoshinori Kobayashi,et al.  Multiple Object Category Detection and Localization Using Generative and Discriminative Models , 2009, IEICE Trans. Inf. Syst..

[19]  Christof Koch,et al.  Saliency on a chip: a digital approach with an FPGA , 2004 .

[20]  Andrea Vedaldi,et al.  Objects in Context , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  Dirk B. Walther,et al.  Saliency on a chip , 2007 .