Attentive visual tracking and trajectory estimation for dynamic scene segmentation

Intelligent Co-Pilot Systems (ICPS) offer the next challenge to vehicle-highway automation. The key to ICPSs is the detection of moving objects (other vehicles) from the moving observer using a visual sensor. The aim of the work presented in this thesis was to design and implement a feature detection and tracking strategy that is capable of tracking image features independently, in parallel, and in real-time and to cluster/segment features utilising the inherent temporal information contained within feature trajectories. Most images contain areas that are of little or no interest to vision tasks. An attentive, data-driven, approach to feature detection and tracking is proposed which aims to increase the efficiency of feature detection and tracking by focusing attention onto relevant regions of the image likely to contain scene structure. This attentive algorithm lends itself naturally to parallelisation and results from a parallel implementation are presented. A scene may be segmented into independently moving objects based on the assumption that features belonging to the same object will move in an identical way in three dimensions (this assumes objects are rigid). A model for scene segmentation is proposed that uses information contained within feature trajectories to cluster, or group, features into independently moving objects. This information includes: image-plane position, time-to-collision of a feature with the image-plane, and the type of motion observed. The Multiple Model Adaptive Estimator (MMAE) algorithm is extended to cope with constituent filters with different states (MMAE2) in an attempt to accurately estimate the time-to-collision of a feature and provide a reliable idea of the type of motion observed (in the form of a model belief measure). Finally, poor state initialisation is identified as a likely prime cause for poor Extended Kalman Filter (EKF) performance (and hence poor MMAE2 performance) when using high order models. The idea of the neurofuzzy initialised EKF (NF-EKF) is introduced which attempts to reduce the time for an EKF to converge by improving the accuracy of the EKF's initial state estimates.

[1]  D. Magill.  Optimal adaptive estimation of sampled stochastic processes , 1965 .

[2]  Michael J. Flynn,et al.  Very high-speed computing systems , 1966 .

[3]  N. F. Toda,et al.  Divergence in the Kalman Filter , 1967 .

[4]  Michael J. Flynn,et al.  Some Computer Organizations and Their Effectiveness , 1972, IEEE Transactions on Computers.

[5]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[6]  David N. Lee,et al.  A Theory of Visual Control of Braking Based on Information about Time-to-Collision , 1976, Perception.

[7]  J. L. Armstrong.  Programming a Parallel Computer for Robot Vision , 1978, Comput. J..

[8]  S. M. Bozic.  Digital and Kalman filtering , 1979 .

[9]  Hans P. Moravec.  Visual Mapping by a Robot Rover , 1979, IJCAI.

[10]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[11]  William B. Thompson,et al.  Disparity Analysis of Images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  R. Woodham,et al.  Determining the movement of objects from a sequence of images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[14]  Shahid H. Bokhari,et al.  On the Mapping Problem , 1981, IEEE Transactions on Computers.

[15]  Azriel Rosenfeld,et al.  Gray-level corner detection , 1982, Pattern Recognit. Lett..

[16]  Dharma P. Agrawal,et al.  A Pipelined Pseudoparallel System Architecture for Real-Time Dynamic Scene Analysis , 1982, IEEE Transactions on Computers.

[17]  Ramesh C. Jain,et al.  Direct Computation of the Focus of Expansion , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Wang Tang,et al.  Application of multiple model estimation to a recursive terrain height correlation system , 1983 .

[19]  H. Nagel,et al.  On the Selection of Critical Points and Local Curvature Extrema of Region Boundaries for Interframe Matching , 1983 .

[20]  Ramesh C. Jain,et al.  Segmentation of Frame Sequences Obtained by a Moving Observer , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[22]  Narendra Ahuja,et al.  Multiprocessor Pyramid Architectures for Bottom-Up Image Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Peter Maybeck,et al.  Investigation of moving-bank multiple model adaptive algorithms , 1985, 1985 24th IEEE Conference on Decision and Control.

[24]  Rama Chellappa,et al.  Estimation of Object Motion Parameters from Noisy Images , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Ralf Kories,et al.  Image Sequence Processing for the Derivation of Parameters for the Guidance of Mobile Robots , 1986, IAS.

[26]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Christopher G. Harris,et al.  Determination of Ego-Motion from Matched Points , 1987, Alvey Vision Conference.

[28]  Fernando Vicuna,et al.  Associative network applications to low-level machine vision. , 1987, Applied optics.

[29]  J. Alison Noble,et al.  Finding Corners , 1988, Alvey Vision Conference.

[30]  Azriel Rosenfeld.  Parallel Algorithms For Real-Time Vision , 1987, Photonics West - Lasers and Applications in Science and Engineering.

[31]  Peter Maybeck,et al.  Substanitial enhancements to a multiple model adaptive estimator for target image tracking , 1987, 26th IEEE Conference on Decision and Control.

[32]  Ishwar K. Sethi,et al.  Finding Trajectories of Feature Points in a Monocular Image Sequence , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Takeo Kanade,et al.  Vision and Navigation for the Carnegie-Mellon Navlab , 1987 .

[34]  S. D. Yanowitz,et al.  A new method for image segmentation , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[35]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[36]  Dana H. Ballard,et al.  The Rochester Robot , 1988 .

[37]  Hirochika Inoue,et al.  Robotic Attention Processing And Its Application To Visual Guidance , 1988, Other Conferences.

[38]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[39]  Peter J. Burt,et al.  Smart sensing within a pyramid vision machine , 1988, Proc. IEEE.

[40]  R. Bajcsy.  Active perception , 1988, Proc. IEEE.

[41]  Quentin F. Stout,et al.  Mapping vision algorithms to parallel architectures , 1988, Proc. IEEE.

[42]  G. Kitagawa,et al.  Akaike Information Criterion Statistics , 1988 .

[43]  M. Maresca,et al.  Parallel architectures for vision , 1988 .

[44]  Christopher G. Harris,et al.  3D positional integration from image sequences , 1988, Image Vis. Comput..

[45]  B. G. Batchelor,et al.  An Investigation Into Intermediate-Level Image Processing Using A Tree-Connected Transputer Network , 1989, Other Conferences.

[46]  Alessandro Verri,et al.  Motion Segmentation from Optical Flow , 1989, Alvey Vision Conference.

[47]  B. G. Batchelor,et al.  Parallel Algorithms For Image Processing: Their Implementation On A Transputer Network , 1989, Other Conferences.

[48]  J. R. Buchanan.  Automatic target recognition on the connection machine , 1989 .

[49]  J. F. Vega-Riveros,et al.  Review of motion analysis techniques , 1989 .

[50]  Debra Charnley,et al.  Surface reconstruction from outdoor image sequences , 1989, Image Vis. Comput..

[51]  Janak H. Patel,et al.  Point matching in a time sequence of stereo image pairs and its parallel implementation on a multiprocessor , 1989, [1989] Proceedings. Workshop on Visual Motion.

[52]  Janak H. Patel,et al.  Load balancing and task decomposition techniques for parallel implementation of integrated vision systems algorithms , 1989, Proceedings of the 1989 ACM/IEEE Conference on Supercomputing (Supercomputing '89).

[53]  D. Charnley,et al.  Outdoor vehicle navigation using passive 3D vision , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  Susanne E. Hambrusch,et al.  Parallel Algorithms for Line Detection on a Mesh , 1989, J. Parallel Distributed Comput..

[55]  James J. Little,et al.  Algorithmic Techniques for Computer Vision on a Fine-Grained Parallel Machine , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  E. D. Dickmans,et al.  Subject-object discrimination in 4D dynamic scene interpretation for machine vision , 1989, [1989] Proceedings. Workshop on Visual Motion.

[57]  Chris C. Bowman,et al.  Parallel Implementation Of Vision Algorithms On A Hybrid Pipelined/Multitransputer Architecture , 1990, Other Conferences.

[58]  Hamid R. Arabnia,et al.  A Parallel Algorithm for the Arbitrary Rotation of Digitized Images Using Process-and-Data-Decomposition Approach , 1990, J. Parallel Distributed Comput..

[59]  Susan Hackwood,et al.  Attentive sensing strategy for a multiwindow vision architecture , 1990, Other Conferences.

[60]  Sartaj Sahni,et al.  Image Template Matching on MIMD Hypercube Multicomputers , 1990, J. Parallel Distributed Comput..

[61]  Yiannis Aloimonos,et al.  Purposive and qualitative active vision , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[62]  Anthony J. Maeder,et al.  Image segmentation for complex natural scenes , 1990, Image Vis. Comput..

[63]  Rama Chellappa,et al.  3-D Motion Estimation Using a Sequence of Noisy Stereo Images: Models, Estimation, and Uniqueness Results , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  John Porrill,et al.  Parallel 3D vision for vehicle navigation and control , 1991 .

[65]  Kwangyoen Wohn,et al.  Recovery of 3D motion of a single particle , 1991, Pattern Recognit..

[66]  Alok N. Choudhary,et al.  Implementation and Evaluation of Hough Transform Algorithms on a Shared-Memory Multiprocessor , 1991, J. Parallel Distributed Comput..

[67]  N. Ranganathan,et al.  A VLSI architecture for dynamic scene analysis , 1991, CVGIP Image Underst..

[68]  Andrew M. Wallace,et al.  Parallel algorithms for plane detection using an adaptive Hough transform , 1991, Image Vis. Comput..

[69]  E T. Leighton,et al.  Introduction to parallel algorithms and architectures , 1991 .

[70]  V. Gengenbach,et al.  A versatile parallel computer architecture for machine vision , 1991 .

[71]  Neil A. Thacker,et al.  Realtime tracking and identification of multiple objects in a cluttered environment , 1991 .

[72]  Christopher G. Harris.  Structure-from-motion under orthographic projection , 1991, Image Vis. Comput..

[73]  Ramesh C. Jain,et al.  A Parallel Technique for Signal-Level Perceptual Organization , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Wilfried Enkelmann,et al.  Obstacle detection by evaluation of optical flow fields from image sequences , 1990, Image Vis. Comput..

[75]  F. Broqua.  Impact of automatic and semi-automatic vehicle longitudinal control on motorway traffic , 1992, Proceedings of the Intelligent Vehicles `92 Symposium.

[76]  Alok Choudhary,et al.  Parallel implementation and evaluation of a motion estimation system algorithm using several data decomposition strategies , 1992 .

[77]  Narendra Ahuja,et al.  Motion and Structure Factorization and Segmentation of Long Multiple Motion Image Sequences , 1992, ECCV.

[78]  Mubarak Shah,et al.  Interpretation of Motion Trajectories using Focus of Expansion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Larry S. Shapiro,et al.  A Matching and Tracking Strategy for Independently Moving Objects , 1992 .

[80]  Stephen M. Smith,et al.  A New Class of Corner Finder , 1992, BMVC.

[81]  S. E. Shladover.  The California PATH Program of IVHS research and its approach to vehicle-highway automation , 1992, Proceedings of the Intelligent Vehicles `92 Symposium.

[82]  Eivind J. Lund,et al.  Multiple model estimation with inter-residual distance feedback , 1992 .

[83]  D. Coombs.  Real-Time Gaze Holding in Binocular Robot Vision , 1992 .

[84]  Richard James Walker.  Multi Sensor Data Fusion Applied to a Class of Autonomous Land Vehicles , 1993 .

[85]  S. Chiu,et al.  On-Line Detection & Correction of Kalman Filter Divergence by Fuzzy Logic , 1993, 1993 American Control Conference.

[86]  B. T. Thomas,et al.  Navigation of An Autonomous Road Vehicle Using Lane Boundary Markings , 1993 .

[87]  T. Kavli.  ASMO—Dan algorithm for adaptive spline modelling of observation data , 1993 .

[88]  Allen R. Hanson,et al.  Image Description and 3-D Reconstruction From Image Trajectories of Rotational Motion , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  E. Rogers,et al.  Utilising fuzzy models in the design of estimators and predictors: an agile target tracking example , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[90]  Steven A. Shafer,et al.  What is the center of the image? , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[91]  Majid Mirmehdi,et al.  Parallel approach to tracking edge segments in dynamic scenes , 1993, Image Vis. Comput..

[92]  W. Dale Blair,et al.  Tracking maneuvering targets with multiple sensors using the interacting multiple model algorithm , 1993, Defense, Security, and Sensing.

[93]  Josef Kittler,et al.  Vanishing point detection , 1993, Image Vis. Comput..

[94]  Martin Brown,et al.  Neurofuzzy adaptive modelling and control , 1994 .

[95]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[96]  Mubarak Shah,et al.  Motion segmentation and estimation , 1994, Proceedings of 1st International Conference on Image Processing.

[97]  Martin Brown,et al.  Theoretical aspects of the CMAC and its application to high-dimensional aerospace modelling problems , 1994 .

[98]  Barbara F. La Scala,et al.  Design of an extended Kalman filter frequency tracker , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[99]  W.C. Collier,et al.  Smart cars, smart highways , 1994, IEEE Spectrum.

[100]  J. M. Brady,et al.  A scene segmenter; visual tracking of moving vehicles , 1994 .

[101]  M. West.  Safety and Social Aspects of Intelligent Vehicle-highway Systems , 1994 .

[102]  M. Shah,et al.  A general approach for determining 3D motion and structure of multiple objects from image trajectories , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[103]  Li-Xin Wang,et al.  Adaptive fuzzy systems and control , 1994 .

[104]  Chris J. Harris,et al.  Multisensor data fusion for obstacle tracking using neuro-fuzzy estimation algorithms , 1994, Defense, Security, and Sensing.