Low dimensional vs point-wise complex object detection, refinement and tracking

In this dissertation, I focus on algorithms to detect, refine, and track complex visual objects, as determined by their shape, appearance, or range of motion. Two different classes of representations, namely low-dimensional and point-wise (nonparametric), are explored, compared, and analyzed by combining multiple features from both imaging and depth sensors. Such perception modules may provide crucial decision-making and planning information for intelligent systems, and we show links to several robotic application areas. On one side, a multimodal human detection algorithm which utilizes multiple sensor data sources is detailed. The human is represented in a low-dimensional space, and a ladar-camera architecture is constructed which with visual and geometric cues improves on the detection rate and speed of conventional human classifiers. Unlike existing approaches, the proposed human classifier does not make any restrictive assumptions on the range scan positions, and thus is applicable to a wide range of real-life detection tasks. On another side, a modified graph cut-based method is proposed to refine complex objects–e.g. human hands, hiking trails for navigation, and household objects– from a rough estimate of the object pose given by a low-dimensional shape detector or tracker. The standard graph cut method is modified to incorporate color and shape distance terms, adaptively weighting them at run time to favor the most informative cue in different visual conditions. Also, this method is extended for point-wise tracking of the objects through iterative refinement without estimation of the object displacement over time. Moreover, a novel algorithm which combines lowand high-level observations in a graph cut framework is developed for the purpose of refining low-dimensional