SMOR: A Semantic Multi-View Object Representation System in 2D Image Sequences

In this work, we propose a framework for segmenting and tracking objects in a static scene using 2D image sequences taken from different viewpoints. The system includes the use of a set of simple algorithms to segment, identify, group and track the semantic objects modeled as a set of regions with compatible surface features. The integration of these algorithms for design of the system is the main contribution of our work. Previous works have considered this issue for the sequences of the static scenes containing a single object or dynamic scenes of multiple objects from a single shot. We are studying this issue for multiple objects from different shots. We present a hierarchical segmentation of image sequences into shots, regions, and objects. Shots are detected using inter-frame information related to camera motion, the regions are identified using the color information in each frame, and the objects are formed by adjacent regions with compatible shape and form. Then we track the segmented regions/objects throughout the sequence using motion information and color constancy. The result of the system is a multi-view representation of objects of a static scene. Comparisons to demonstrate the performance of the systems is provided and discussed.