A reinforcement agent for object segmentation in ultrasound images

The principal contribution of this work is to design a general framework for an intelligent system to extract one object of interest from ultrasound images. This system is based on reinforcement learning. The input image is divided into several sub-images, and the proposed system finds the appropriate local values for each of them so that it can extract the object of interest. The agent uses some images and their ground-truth (manually segmented) version to learn from. A reward function is employed to measure the similarities between the output and the manually segmented images, and to provide feedback to the agent. The information obtained can be used as valuable knowledge stored in the Q-matrix. The agent can then use this knowledge for new input images. The experimental results for prostate segmentation in trans-rectal ultrasound images show high potential of this approach in the field of ultrasound image segmentation.

[1]  Robert J. Holt,et al.  Using line correspondences in invariant signatures for curve recognition , 1993, Image Vis. Comput..

[2]  A Fenster,et al.  Prostate boundary segmentation from 2D ultrasound images. , 2000, Medical physics.

[3]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[4]  Hamid R. Tizhoosh,et al.  Filter fusion for image enhancement using reinforcement learning , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[5]  Hamid R. Tizhoosh,et al.  Increasing Object Recognition Rate using Reinforced Segmentation , 2006, 2006 International Conference on Image Processing.

[6]  Abraham Kandel,et al.  3-Dimensional curve similarity using string matching , 2004, Robotics Auton. Syst..

[7]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[8]  J. S. Prater,et al.  Segmenting ultrasound images of the prostate using neural networks. , 1992, Ultrasonic imaging.

[9]  Hamid R. Tizhoosh,et al.  Using Reinforcement Learning for Filter Fusion in Image Enhancement , 2005, Computational Intelligence.

[10]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[11]  Aaron Fenster,et al.  Semiautomatic three-dimensional segmentation of the prostate using two-dimensional ultrasound images. , 2003, Medical physics.

[12]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[13]  Graham W. Taylor.  A reinforcement learning framework for parameter control in computer vision applications , 2004, First Canadian Conference on Computer and Robot Vision, 2004. Proceedings..

[14]  Yongmin Kim,et al.  Edge-guided boundary delineation in prostate ultrasound images , 2000, IEEE Transactions on Medical Imaging.

[15]  Farhang Sahba,et al.  A coarse-to-fine approach to prostate boundary segmentation in ultrasound images , 2005, Biomedical engineering online.

[16]  Hamid R. Tizhoosh,et al.  Reinforced Contrast Adaptation , 2006, Int. J. Image Graph..

[17]  Introduction to Reinforcement Learning 1 What is Reinforcement Learning ? , .

[18]  Peng-Yeng Yin,et al.  Maximum entropy-based optimal threshold selection using deterministic reinforcement learning with controlled randomization , 2002, Signal Process..

[19]  Yongmin Kim,et al.  Parametric shape modeling using deformable superellipses for prostate segmentation , 2004, IEEE Transactions on Medical Imaging.

[20]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[21]  Aaron Fenster,et al.  Prostate segmentation by feature enhancement using domain knowledge and adaptive region based operations , 2006, Physics in medicine and biology.

[22]  Hamid R. Tizhoosh,et al.  Q(lambda)-Based Image Thresholding. , 2004 .

[23]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[24]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[25]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[26]  Dinggang Shen,et al.  Segmentation of prostate boundaries from ultrasound images using statistical shape model , 2003, IEEE Transactions on Medical Imaging.

[27]  Maximilien Vermandel,et al.  Segmentation of abdominal ultrasound images of the prostate using a priori information and an adapted noise filter. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[28]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[30]  Hamid R. Tizhoosh,et al.  A Reinforcement Learning Framework for Medical Image Segmentation , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[31]  Robert Simons,et al.  American Cancer Society: Access to Care , 2008 .

[32]  Aaron Fenster,et al.  Prostate segmentation algorithm using dyadic wavelet transform and discrete dynamic contour. , 2004, Physics in medicine and biology.