A Sparse Branch and Bound Optimization of Noisy Weighted DAG Modification Under Constraints: A Method for Monocular Data Association to Multiple Laser Planes

The line segments of multiple (and nearly parallel) planar laser sources appearing in monocular video are objectively associated to their respective laser sources. Such association is a precursor to 3D surface digitization where each laser plane develops a 3D surface. In this work, the detected laser line segments are modeled as a weighted directed acyclic graph (wDAG), in which the graph nodes represent the line segments and their maximum path hopes from any source node represents the laser source label. However, due to noise and occlusions, the structure of the wDAG will normally vary from that expected in the ideal cases. Furthermore, it is essential to robustness to incorporate the association information of the previous frames in the new frame associations. The proposed work objectively modifies the current frame’s wDAG under these constraints so the resulting wDAG becomes in an appropriate structure for the labeling of the detected line segments. Branch and bound (BnB) optimization has been a good choice for global optimization of combinatorial problems. Rather than approaching the optimization as a pure (or full) combinatorial problem, the proposed prioritizations of the branch modification actions are nearly optimal. The high priority actions will very likely lead to an optimal modification. This aspect is exploited by the proposed sparse variant of BnB (SBnB) optimization. It significantly reduces the number of the branches to be considered, in addition to those contributed by the bounding part of the BnB optimization. The proposed approach was tested on numerous monocular videos and plausible association results were efficiently achieved.

[1]  Henk A. P. Blom,et al.  Probabilistic data association avoiding track coalescence , 2000, IEEE Trans. Autom. Control..

[2]  Muhammad Sualeh,et al.  Dynamic Multi-LiDAR Based Multiple Object Detection and Tracking , 2019, Sensors.

[3]  A. Breskin,et al.  Directed Acyclic Graphs, Effect Measure Modification, and Generalizability. , 2020, American journal of epidemiology.

[4]  Joaquim Salvi,et al.  A state of the art in structured light patterns for surface profilometry , 2010, Pattern Recognit..

[5]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Peter Willett,et al.  Set JPDA algorithm for tracking unordered sets of targets , 2009, 2009 12th International Conference on Information Fusion.

[7]  Jake K. Aggarwal,et al.  Binocular versus trinocular stereo , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[8]  Andrew W. Moore,et al.  Finding optimal Bayesian networks by dynamic programming , 2005 .

[9]  Romaric Audigier,et al.  Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking , 2016, ECCV.

[10]  Sean L. Bowman,et al.  Probabilistic data association for semantic SLAM , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[11]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[12]  Amit K. Roy-Chowdhury,et al.  Distributed Multi-Target Tracking and Data Association in Vision Networks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Kobra Etminani,et al.  Effective pruning strategies for branch and bound Bayesian networks structure learning from data , 2013, Sci. Iran..

[15]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[16]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[17]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[18]  Takayuki Kanda,et al.  Person Tracking in Large Public Spaces Using 3-D Range Sensors , 2013, IEEE Transactions on Human-Machine Systems.

[19]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[20]  杜华,et al.  Multi-line array laser three-dimensional scanning system, and multi-line array laser three-dimensional scanning method , 2016 .

[21]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[22]  Paolo Braca,et al.  A Scalable Algorithm for Tracking an Unknown Number of Targets Using Multiple Sensors , 2016, IEEE Transactions on Signal Processing.

[23]  Yue Wang,et al.  2D Laser SLAM With General Features Represented by Implicit Functions , 2020, IEEE Robotics and Automation Letters.

[24]  Qiang Ji,et al.  Efficient Structure Learning of Bayesian Networks using Constraints , 2011, J. Mach. Learn. Res..

[25]  Lei Huang,et al.  Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review , 2016 .

[26]  Ling Shao,et al.  Recent advances and trends in visual tracking: A review , 2011, Neurocomputing.

[27]  Erica Nocerino,et al.  Dense image matching: Comparisons and analyses , 2013, 2013 Digital Heritage International Congress (DigitalHeritage).

[28]  Fakhri Karray,et al.  Multisensor data fusion: A review of the state-of-the-art , 2013, Inf. Fusion.

[29]  Federico Castelletti,et al.  Bayesian Model Selection of Gaussian Directed Acyclic Graph Structures , 2020, International Statistical Review.

[30]  Shuai Liu,et al.  A review of visual moving target tracking , 2017, Multimedia Tools and Applications.

[31]  Henry Leung,et al.  Data fusion in intelligent transportation systems: Progress and challenges - A survey , 2011, Inf. Fusion.

[32]  Ingemar J. Cox,et al.  A review of statistical data association techniques for motion correspondence , 1993, International Journal of Computer Vision.

[33]  Y. Bar-Shalom Tracking and data association , 1988 .

[34]  Sevket Gümüstekin,et al.  Calibration of double stripe 3D laser scanner systems using planarity and orthogonality constraints , 2014, Digit. Signal Process..

[35]  Tommi S. Jaakkola,et al.  Learning Bayesian Network Structure using LP Relaxations , 2010, AISTATS.

[36]  Shiuh-Ku Weng,et al.  Video object tracking using adaptive Kalman filter , 2006, J. Vis. Commun. Image Represent..

[37]  Diego Klabjan,et al.  Bayesian Network Learning via Topological Order , 2017, J. Mach. Learn. Res..

[38]  Yulan Guo,et al.  Parallax Attention for Unsupervised Stereo Correspondence Learning , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Emilio J. Almazan,et al.  Tracking People across Multiple Non-overlapping RGB-D Sensors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[40]  Daniel Cremers,et al.  Real-time human motion tracking using multiple depth cameras , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Ian D. Reid,et al.  Joint Probabilistic Data Association Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[42]  B. V. K. Vijaya Kumar,et al.  A multi-sensor fusion system for moving object detection and tracking in urban driving environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[44]  Ryo Furukawa,et al.  Self-Calibration of Multiple Laser Planes for 3D Scene Reconstruction , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).