3D Perception With Slanted Stixels on GPU

This article presents a GPU-accelerated software design of the recently proposed model of Slanted Stixels, which represents the geometric and semantic information of a scene in a compact and accurate way. We reformulate the measurement depth model to reduce the computational complexity of the algorithm, relying on the confidence of the depth estimation and the identification of invalid values to handle outliers. The proposed massively parallel scheme and data layout for the irregular computation pattern that corresponds to a Dynamic Programming paradigm is described and carefully analyzed in performance terms. Performance is shown to scale gracefully on current generation embedded GPUs. We assess the proposed methods in terms of semantic and geometric accuracy as well as run-time performance on three publicly available benchmark datasets. Our approach achieves real-time performance with high accuracy for 2048 × 1024 image sizes and 4 × 4 Stixel resolution on the low-power embedded GPU of an NVIDIA Tegra Xavier.

[1]  Markus Enzweiler,et al.  Efficient Stixel-based object recognition , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[2]  Alistair Sutherland,et al.  Fast Obstacle Detection Using Sparse Edge-Based Disparity Maps , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[3]  Dariu M. Gavrila,et al.  Instance Stixels: Segmenting and Grouping Stixels into Objects , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[4]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Luc Van Gool,et al.  Stixels estimation without depth map computation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  Marc Pollefeys,et al.  The Stixel World: A medium-level representation of traffic scenes , 2017, Image Vis. Comput..

[7]  Uwe Franke,et al.  Towards a Global Optimal Multi-Layer Stixel Representation of Dense 3D Data , 2011, BMVC.

[8]  Frank Dellaert,et al.  Modern MAP inference methods for accurate and fast occupancy grid mapping on higher order factor graphs , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Marc Pollefeys,et al.  Semantic Stixels: Depth is not enough , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[10]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[11]  Mark J. Harris,et al.  Parallel Prefix Sum (Scan) with CUDA , 2011 .

[12]  Antonio M. López,et al.  Embedded Real-time Stereo Estimation via Semi-Global Matching on the GPU , 2016, ICCS.

[13]  Klaus C. J. Dietmayer,et al.  Fusion of laser and radar sensor data with a sequential Monte Carlo Bayesian occupancy filter , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[14]  Antonio M. López,et al.  Slanted Stixels: A Way to Represent Steep Streets , 2019, International Journal of Computer Vision.

[15]  Oana Ignat.  Disparity image segmentation for free-space detection , 2016, 2016 IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP).

[16]  Ethan Fetaya,et al.  StixelNet: A Deep Convolutional Network for Obstacle Detection and Road Segmentation , 2015, BMVC.

[17]  David Vázquez,et al.  GPU-Accelerated Real-Time Stixel Computation , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Hui Xiong,et al.  A new benchmark for vision-based cyclist detection , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[19]  Uwe Franke,et al.  The Stixel World - A Compact Medium Level Representation of the 3D-World , 2009, DAGM-Symposium.

[20]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  David Pfeiffer,et al.  The Stixel World: a compact medium-level represantation for efficiently modeling dynamic three-dimensional environments , 2011 .

[22]  Stefan Roth,et al.  Object-Level Priors for Stixel Generation , 2014, GCPR.

[23]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Nicolai Schneider,et al.  Stix-Fusion: A Probabilistic Stixel Integration Technique , 2014, 2014 Canadian Conference on Computer and Robot Vision.

[25]  David Pfeiffer,et al.  Modeling Dynamic 3D Environments by Means of The Stixel World , 2011, IEEE Intelligent Transportation Systems Magazine.

[26]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[27]  Stefan K. Gehrig,et al.  Exploiting the Power of Stereo Confidences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Stefan Roth,et al.  Stixmantics: A Medium-Level Model for Real-Time Semantic Scene Understanding , 2014, ECCV.

[29]  Marc Pollefeys,et al.  Slanted Stixels: Representing San Francisco's Steepest Streets , 2017, BMVC.

[30]  Uwe Franke,et al.  Stixmentation - Probabilistic Stixel based Traffic Scene Labeling , 2012, BMVC.

[31]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  Luc Van Gool,et al.  Fast Stixel Computation for Fast Pedestrian Detection , 2012, ECCV Workshops.