Recognition of Moving Object in High Dynamic Scene for Visual Prosthesis

Currently, visual perceptions generated by visual prosthesis are low resolution with unruly color and restricted grayscale. This severely restricts the ability of prosthetic implant to complete visual tasks in daily scenes. Some studies explore existing image processing techniques to improve the percepts of objects in prosthetic vision. However, most of them extract the moving objects and optimize the visual percepts in general dynamic scenes. The application of visual prosthesis in daily life scenes with high dynamic is greatly limited. Hence, in this study, a novel unsupervised moving object segmentation model is proposed to automatically extract the moving objects in high dynamic scene. In this model, foreground cues with spatiotemporal edge features and background cues with boundary-prior are exploited, the moving object proximity map are generated in dynamic scene according to the manifold ranking function. Moreover, the foreground and background cues are ranked simultaneously, and the moving objects are extracted by the two ranking maps integration. The evaluation experiment indicates that the proposed method can uniformly highlight the moving object and keep good boundaries in high dynamic scene with other methods. Based on this model, two optimization strategies are proposed to improve the perception of moving objects under simulated prosthetic vision. Experimental results demonstrate that the introduction of optimization strategies based on the moving object segmentation model can efficiently segment and enhance moving objects in high dynamic scene, and significantly improve the recognition performance of moving objects for the blind. key words: visual prosthesis, moving object segmentation, high dynamic scene, prosthetic vision

[1]  J. Sahel,et al.  Subjects Blinded By Outer Retinal Dystrophies Are Able To Recognize Outlined Shapes Using The Argus® Ii Retinal Prosthesis System: A Comparison With The Full Shapes Recognition Task , 2012 .

[2]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Ming-Hsuan Yang,et al.  SegFlow: Joint Learning for Video Object Segmentation and Optical Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Heng Li,et al.  Image processing strategies based on saliency segmentation for object recognition under simulated prosthetic vision , 2017, Artif. Intell. Medicine.

[5]  M. Humayun,et al.  Artificial vision through neuronal stimulation , 2012, Neuroscience Letters.

[6]  Anthony J. Maeder,et al.  Region-of-interest processing for electronic visual prostheses , 2008, J. Electronic Imaging.

[7]  Michael J. Black,et al.  A Fully-Connected Layered Model of Foreground and Background Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Nick Barnes,et al.  Time-to-contact maps for navigation with a low resolution visual prosthesis , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[10]  Luc Van Gool,et al.  One-Shot Video Object Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Bohyung Han,et al.  Generalized background subtraction based on hybrid inference by belief propagation and Bayesian filtering , 2011, 2011 International Conference on Computer Vision.

[12]  Yanyu Lu,et al.  Image processing based recognition of images with a limited number of pixels using simulated prosthetic vision , 2010, Inf. Sci..

[13]  B. Wilhelm,et al.  Subretinal Visual Implant Alpha IMS – Clinical trial interim report , 2015, Vision Research.

[14]  E. Zrenner,et al.  The Subretinal Implant ALPHA: Implantation and Functional Results , 2017 .

[15]  Y. Luo,et al.  A review and update on the current status of retinal prostheses (bionic eye). , 2014, British medical bulletin.

[16]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[17]  Junzhou Huang,et al.  Background Subtraction Using Low Rank and Group Sparsity Constraints , 2012, ECCV.

[18]  Heng Li,et al.  A real-time image optimization strategy based on global saliency detection for artificial retinal prostheses , 2017, Inf. Sci..

[19]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jen-Hui Chuang,et al.  Regularized Background Adaptation: A Novel Learning Rate Control Scheme for Gaussian Mixture Modeling , 2011, IEEE Transactions on Image Processing.

[21]  Matthew Berger,et al.  Subspace Tracking under Dynamic Dimensionality for Online Background Subtraction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Jessy D. Dorn,et al.  Interim results from the international trial of Second Sight's visual prosthesis. , 2012, Ophthalmology.

[23]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[24]  Yuan Yang,et al.  Optimization of Visual Information Presentation for Visual Prosthesis , 2018, Int. J. Biomed. Imaging.

[25]  Anthony J. Maeder,et al.  Challenges in digital imaging for artificial human vision , 2001, IS&T/SPIE Electronic Imaging.

[26]  Jitendra Malik,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Segmentation of Moving Objects by Long Term Video Analysis , 2022 .

[27]  E. Maynard,et al.  Visual prostheses. , 2001, Annual review of biomedical engineering.

[28]  Erik G. Learned-Miller,et al.  It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videos , 2016, ECCV.

[29]  S. Süsstrunk,et al.  SLIC Superpixels ? , 2010 .

[30]  Toby Sharp,et al.  Image segmentation with a bounding box prior , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[31]  E. Zrenner Will Retinal Implants Restore Vision ? , 2002 .

[32]  James W. Davis,et al.  A Multi-transformational Model for Background Subtraction with Moving Cameras , 2014, ECCV.

[33]  Jie Hu,et al.  Optimizing Chinese character displays improves recognition and reading performance of simulated irregular phosphene maps. , 2013, Investigative ophthalmology & visual science.

[34]  Allen R. Hanson,et al.  Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Stephen L Hicks,et al.  Simulating prosthetic vision: Optimizing the information content of a limited visual display. , 2010, Journal of vision.

[36]  Chris E. Williams,et al.  Visual prostheses for the blind. , 2013, Trends in biotechnology.

[37]  A Treisman,et al.  Feature analysis in early vision: evidence from search asymmetries. , 1988, Psychological review.

[38]  Truong Q. Nguyen,et al.  Moving Object Detection With a Freely Moving Camera via Background Motion Subtraction , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  James D. Weiland,et al.  Retinal stimulation strategies to restore vision: Fundamentals and systems , 2016, Progress in Retinal and Eye Research.

[40]  Fatih Murat Porikli,et al.  Saliency-aware geodesic video object segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Gislin Dagnelie,et al.  Visually guided performance of simple tasks using simulated prosthetic vision. , 2003, Artificial organs.

[42]  V. C. Coffey Vision Accomplished: The Bionic Eye , 2017 .

[43]  Nikos Paragios,et al.  Motion-based background subtraction using adaptive kernel density estimation , 2004, CVPR 2004.

[44]  N Parikh,et al.  Saliency-based image processing for retinal prostheses , 2010, Journal of neural engineering.

[45]  G. Suaning,et al.  Attaining higher resolution visual prosthetics: a review of the factors and limitations , 2013, Journal of neural engineering.

[46]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[47]  Karteek Alahari,et al.  Learning Motion Patterns in Videos , 2016, CVPR.

[48]  Kurt Keutzer,et al.  Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow , 2010, ECCV.

[49]  Yitzhak Yitzhaky,et al.  Active confocal imaging for visual prostheses , 2015, Vision Research.

[50]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[51]  Marc Van Droogenbroeck,et al.  ViBe: A Universal Background Subtraction Algorithm for Video Sequences , 2011, IEEE Transactions on Image Processing.

[52]  Li Xu,et al.  A Segmentation Based Variational Model for Accurate Optical Flow Estimation , 2008, ECCV.

[53]  Liming Li,et al.  Image Processing Strategies Based on a Visual Saliency Model for Object Recognition Under Simulated Prosthetic Vision. , 2016, Artificial organs.

[54]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Yanyu Lu,et al.  Moving object recognition under simulated prosthetic vision using background-subtraction-based image processing strategies , 2014, Inf. Sci..

[56]  Ahmed M. Elgammal,et al.  Online Moving Camera Background Subtraction , 2012, ECCV.

[57]  Jongin Lim,et al.  Scene conditional background update for moving object detection in a moving camera , 2017, Pattern Recognit. Lett..

[58]  Neeta Nain,et al.  An Adaptive Codebook Model for Change Detection with Dynamic Background , 2015, 2015 11th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[59]  Gislin Dagnelie,et al.  Facial recognition using simulated prosthetic pixelized vision. , 2003, Investigative ophthalmology & visual science.

[60]  Ahmed M. Elgammal,et al.  A Multilayer-Based Framework for Online Background Subtraction with Freely Moving Cameras , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).