Toward large-area mosaicing for underwater scientific applications

Severe attenuation and backscatter of light fundamentally limits our ability to image extended underwater scenes. Generating a composite view or mosaic from multiple overlapping images is usually the most practical and flexible way around this limitation. In this paper, we look at the general constraints associated with imaging from underwater vehicles for scientific applications - low overlap, nonuniform lighting, and unstructured motion $and present a methodology for dealing with these constraints toward a solution of the problem of large-area global mosaicing. Our approach assumes that the extended scene is planar and determines the homographies for each image by estimating and compensating for radial distortion, topology estimation through feature-based pairwise image registration using a multiscale Harris interest point detector coupled with a feature descriptor based on Zernike moments, and global registration across all images based on the initial registration derived from the pairwise estimates. This approach is purely image based and does not assume that navigation data is available. We demonstrate the utility of our techniques using real data obtained using the Jason remotely operated vehicle (ROV) at an archaeological site covering hundreds of square meters.

[1]  Whoi-Yul Kim,et al.  Robust Rotation Angle Estimator , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Harpreet S. Sawhney,et al.  Automated Mosaics via Topology Inference , 2002, IEEE Computer Graphics and Applications.

[3]  Jonathan C. Howland.  Digital Data Logging and Processing, Derbyshire Survey, 1997 , 1999 .

[4]  Andrew Calway,et al.  Image Registration using Multiresolution Frequency Domain Correlation , 1998, BMVC.

[5]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[6]  Philip F. McLauchlan,et al.  Image mosaicing using sequential bundle adjustment , 2002, Image Vis. Comput..

[7]  Gideon P. Stein,et al.  Lens distortion calibration using point correspondences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  C. Smith.  Whale falls: chemosynthesis on the deep seafloor , 1992 .

[9]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[10]  Shahriar Negahdaripour,et al.  Direct estimation of motion from sea floor images for automatic station-keeping of submersible platforms , 1999 .

[11]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Reinhard Koch,et al.  Hand-held acquisition of 3D models with a video camera , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[13]  R. L. Marks,et al.  Improved real-time videomosaicking of the ocean floor , 1995, 'Challenges of Our Changing Global Environment'. Conference Proceedings. OCEANS '95 MTS/IEEE.

[14]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[15]  B. N. Chatterji,et al.  An FFT-based technique for translation, rotation, and scale-invariant image registration , 1996, IEEE Trans. Image Process..

[16]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[17]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[18]  Heung-Yeung Shum,et al.  Panoramic Image Mosaics , 1998 .

[19]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[20]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[21]  Zhengyou Zhang,et al.  On the epipolar geometry between two images with lens distortion , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[23]  K. Foote.  Censusing marine living resources in the Gulf of Maine: a proposal , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[24]  Dana R. Yoerger,et al.  Fine-scale seafloor survey in rugged deep-ocean terrain with an autonomous robot , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[25]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[26]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[27]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[28]  Harry Shum,et al.  Construction and refinement of panoramic mosaics with global and local alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[30]  Olivier D. Faugeras,et al.  A theory of self-calibration of a moving camera , 1992, International Journal of Computer Vision.

[31]  Michal Irani,et al.  Representation of scenes from collections of images , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[32]  P. Anandan,et al.  Efficient representations of video sequences and their applications , 1996, Signal Process. Image Commun..

[33]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[34]  Richard Szeliski,et al.  Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001 .

[35]  C. Schmid,et al.  Matching by local invariants , 1995 .

[36]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Roland T. Chin,et al.  On Image Analysis by the Methods of Moments , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Harpreet S. Sawhney,et al.  True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Hanumant Singh,et al.  Advances in Doppler-based navigation of underwater robotic vehicles , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[40]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[41]  J. Reynolds,et al.  Fisheries and fisheries habitat investigations using undersea technology , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[42]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[43]  Philip F. McLauchlan,et al.  Image Mosaicing using Sequential Bundle Adjustments , 2000, BMVC.

[44]  Harpreet S. Sawhney,et al.  VideoBrush/sup TM/: experiences with consumer video mosaicing , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[45]  T. Lindeberg,et al.  Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[46]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[47]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[48]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  H. H. Wang,et al.  Video mosaicking along arbitrary vehicle paths , 1996, Proceedings of Symposium on Autonomous Underwater Vehicle Technology.

[50]  Whoi-Yul Kim,et al.  A region-based shape descriptor using Zernike moments , 2000, Signal Process. Image Commun..

[51]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[52]  Gregory Dudek,et al.  Rotation and zooming in image mosaicing , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[53]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[54]  D. Yoerger,et al.  The discovery of ancient history in the deep sea using advanced deep submergence technology , 2000 .

[55]  Seth J. Teller,et al.  Spherical Mosaics with Quaternions and Dense Correlation , 2000, International Journal of Computer Vision.

[56]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[57]  Andrew Zisserman,et al.  Automated mosaicing with super-resolution zoom , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[58]  J. Santos-Victor,et al.  Underwater mosaicing and trajectory reconstruction using global alignment , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[59]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Andrew Zisserman,et al.  Viewpoint invariant texture matching and wide baseline stereo , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[61]  Charles V. Stewart,et al.  A feature-based technique for joint, linear estimation of high-order image-to-mosaic transformations: application to mosaicing the curved human retina , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[62]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[63]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[64]  D. Yoerger,et al.  Iron Age Shipwrecks in Deep Water off Ashkelon, Israel , 2002, American Journal of Archaeology.

[65]  Adam Baumberg,et al.  Reliable feature matching across widely separated views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).