Unsupervised Domain Adaptation Learning Algorithm for RGB-D Staircase Recognition

Detection and recognition of staircase as upstairs, downstairs and negative (e.g., ladder) are the fundamental of assisting the visually impaired to travel independently in unfamiliar environments. Previous researches have focused on using massive amounts of RGB-D scene data to train traditional machine learning (ML) based models to detect and recognize the staircase. However, the performance of traditional ML techniques is limited by the amount of labeled RGB-D staircase data. In this paper, we apply an unsupervised domain adaptation approach in deep architectures to transfer knowledge learned from the labeled RGB-D stationary staircase dataset to the unlabeled RGB-D escalator dataset. By utilizing the domain adaptation method, our feedforward convolutional neural networks (CNN) based feature extractor with 5 convolution layers can achieve 100% classification accuracy on testing the labeled stationary staircase data and 80.6% classification accuracy on testing the unlabeled escalator data. We demonstrate the success of the approach for classifying staircase on two domains with a limited amount of data. To further demonstrate the effectiveness of the approach, we also validate the same CNN model without domain adaptation and compare its results with those of our proposed architecture.

[1]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[2]  Luis Herranz,et al.  Depth CNNs for RGB-D Scene Recognition: Learning from Scratch Better than Transferring from RGB-CNNs , 2017, AAAI.

[3]  Zhengqi Li,et al.  MegaDepth: Learning Single-View Depth Prediction from Internet Photos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[5]  M. Bousbia-Salah,et al.  An Ultrasonic Navigation System for Blind People , 2007, 2007 IEEE International Conference on Signal Processing and Communications.

[6]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, ICCV.

[7]  D. T. Lee,et al.  Unsupervised Feature Learning for RGB-D Image Classification , 2014, ACCV.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Larry S. Davis,et al.  Learning Discriminative Features via Label Consistent Neural Network , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Dong Xu,et al.  Recognizing RGB Images by Learning from RGB-D Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[12]  Shuihua Wang,et al.  Detecting stairs and pedestrian crosswalks for the blind by RGBD camera , 2012, 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops.

[13]  Yihui He Estimated Depth Map Helps Image Classification , 2017, ArXiv.

[14]  Zhi Zhang,et al.  Bag of Tricks for Image Classification with Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Alexei A. Efros,et al.  Unbiased look at dataset bias , 2011, CVPR 2011.

[16]  Yingli Tian,et al.  Depth-aware indoor staircase detection and recognition for the visually impaired , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).