Multi-view calibration from planar motion for video surveillance

We present a technique for the registration of multiple surveillance cameras through the automatic alignment of image trajectories. The algorithm address the problem of recovering the relative pose of several stationary cameras that observe one or more objects in motion. Each camera tracks several objects to produce a set of trajectories in the image. Using a simple calibration procedure, we recover the relative orientation of each camera to the local ground plane in order to projectively unwarp image trajectories onto a nominal plane of correct orientation. Unwarped trajectory curves are then matched by solving for the 3D to 3D rotation, translation, and scale that bring them into alignment. The relative transform between a pair of cameras is derived from the independent camera-to-ground-plane rotations and the plane-to-plane transform computed from matched trajectories. Registration aligns n-cameras with respect to each other in a single camera frame (that of the reference camera). The approach recovers both the epipolar geometry between all cameras and the camera-to-ground rotation for each camera. After calibration, points that are known to lay on a world ground plane can be directly backprojected into each of the camera frames. The algorithm is demonstrated for two-camera and three-camera scenarios by tracking pedestrians as they move through a surveillance area and matching the resulting trajectories.

[1]  Azriel Rosenfeld,et al.  Visual surveillance and monitoring , 1998 .

[2]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Farid Amoozegar,et al.  Neural-network-based target tracking state-of-the-art survey , 1998 .

[4]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[5]  Robert M. Haralick,et al.  Multi-image camera calibration and object-point estimation , 1995, Defense, Security, and Sensing.

[6]  James W. Davis,et al.  Real-time closed-world tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  Alessandro Neri,et al.  Automatic moving object and background separation , 1998, Signal Process..

[9]  Thomas Skordas,et al.  Calibrating a mobile camera , 1990, Image Vis. Comput..

[10]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[11]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[12]  Alex Pentland,et al.  Camera Self-Calibration From One Point Correspondence , 1995 .

[13]  Harit P. Trivedi,et al.  Semi-analytic method for estimating stereo camera geometry from matched points , 1991, Image Vis. Comput..

[14]  H. J. Buiten,et al.  Quality assessment of remote sensing image registration — analysis and testing of control point residuals , 1997 .

[15]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[16]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.