AI Perspectives in Smart Cities and Communities to Enable Road Vehicle Automation and Smart Traffic Control

Smart cities and communities (SCC) constitute a new paradigm in urban development. SCC ideate a data-centered society aimed at improving efficiency by automating and optimizing activities and utilities. Information and communication technology along with Internet of Things enables data collection and with the help of artificial intelligence (AI) situation awareness can be obtained to feed the SCC actors with enriched knowledge. This paper describes AI perspectives in SCC and gives an overview of AI-based technologies used in traffic to enable road vehicle automation and smart traffic control. Perception, smart traffic control and driver modeling are described along with open research challenges and standardization to help introduce advanced driver assistance systems and automated vehicle functionality in traffic. To fully realize the potential of SCC, to create a holistic view on a city level, availability of data from different stakeholders is necessary. Further, though AI technologies provide accurate predictions and classifications, there is an ambiguity regarding the correctness of their outputs. This can make it difficult for the human operator to trust the system. Today there are no methods that can be used to match function requirements with the level of detail in data annotation in order to train an accurate model. Another challenge related to trust is explainability: models can have difficulty explaining how they came to certain conclusions, so it is difficult for humans to trust them.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Fernando Alonso-Fernandez,et al.  Action and Intention Recognition of Pedestrians in Urban Traffic , 2018, 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[3]  Wenshuo Wang,et al.  Cluster Naturalistic Driving Encounters Using Deep Unsupervised Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[4]  Juan-Carlos Cano,et al.  Reducing emergency services arrival time by using vehicular communications and Evolution Strategies , 2014, Expert Syst. Appl..

[5]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[6]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[7]  Zhixin Wang,et al.  Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Eren Erdal Aksoy,et al.  SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds , 2020, ISVC.

[9]  Eren Erdal Aksoy,et al.  SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[10]  G. Ferrari,et al.  IoT-Enabled Smart Sustainable Cities: Challenges and Approaches , 2020, Smart Cities.

[11]  Juan-Carlos Cano,et al.  A UAV-Based Content Delivery Architecture for Rural Areas and Future Smart Cities , 2019, IEEE Internet Computing.

[12]  Carlos Miguel Tavares Calafate,et al.  Crowdsensing and Vehicle-Based Sensing , 2016, Mob. Inf. Syst..

[13]  Yue Zhang,et al.  N-ary Relation Extraction using Graph-State LSTM , 2018, EMNLP.

[14]  Runhe Huang,et al.  Sequential Graph Neural Network for Urban Road Traffic Speed Prediction , 2020, IEEE Access.

[15]  P. Siano,et al.  Iot-based smart cities: A survey , 2016, 2016 IEEE 16th International Conference on Environment and Electrical Engineering (EEEIC).

[16]  Ulf Holmberg,et al.  A Modular CACC System Integration and Design , 2012, IEEE Transactions on Intelligent Transportation Systems.

[17]  Qingquan Li,et al.  A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios , 2014, IEEE Transactions on Vehicular Technology.

[18]  Dariu M. Gavrila,et al.  Human motion trajectory prediction: a survey , 2019, Int. J. Robotics Res..

[19]  Elham Semsar-Kazerooni,et al.  The Grand Cooperative Driving Challenge 2016: boosting the introduction of cooperative automated vehicles , 2016, IEEE Wireless Communications.

[20]  Klaus C. J. Dietmayer,et al.  Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[21]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[22]  Guodong Guo,et al.  A survey on deep learning based face recognition , 2019, Comput. Vis. Image Underst..

[23]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[24]  Akhilesh Kumar Maurya,et al.  Study of Deceleration Behaviour of Different Vehicle Types , 2012 .

[25]  Ramon Bragós,et al.  Ventilation and Heart Rate Monitoring in Drivers using a Contactless Electrical Bioimpedance System , 2013 .

[26]  Yin Zhou,et al.  VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Patrizio Campisi,et al.  Longitudinal Evaluation of EEG-Based Biometric Recognition , 2017, IEEE Transactions on Information Forensics and Security.

[28]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[29]  Kurt Keutzer,et al.  SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Jenn-Jier James Lien,et al.  Automatic Vehicle Detection Using Local Features—A Statistical Approach , 2008, IEEE Transactions on Intelligent Transportation Systems.

[31]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[32]  Rui Zhang,et al.  All In One Network for Driver Attention Monitoring , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  B. D. Ripley,et al.  Neural Networks for Pattern Recognition.@@@Pattern Recognition and Neural Networks. , 1997 .

[34]  F. Alonso-Fernandez,et al.  Master ' s Programme in Embedded and Intelligent Systems , 120 credits Behaviour modelling of Vehicles at a Roundabout , 2020 .

[35]  Anna Niska,et al.  Statistik över cyklisters olyckor: faktaunderlag till gemensam strategi för säker cykling , 2013 .

[36]  Kurt Keutzer,et al.  SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[37]  A. Vinel,et al.  Lonely road: speculative challenges for a social media robot aimed to reduce driver loneliness , 2021, ICWSM Workshops.

[38]  Juan-Carlos Cano,et al.  Crowdsensing in Smart Cities: Overview, Platforms, and Environment Sensing Issues , 2018, Sensors.

[39]  Steffen Leonhardt,et al.  ECG on the Road: Robust and Unobtrusive Estimation of Heart Rate , 2011, IEEE Transactions on Biomedical Engineering.

[40]  Changzhi Li,et al.  A Review on Recent Advances in Doppler Radar Sensors for Noncontact Healthcare Monitoring , 2013, IEEE Transactions on Microwave Theory and Techniques.

[41]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[42]  Alexandre M. Bayen,et al.  Flow: Deep Reinforcement Learning for Control in SUMO , 2018 .

[43]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[44]  Tony Larsson,et al.  Dimensions of cooperative driving, its and automation , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[45]  Alexandre M. Bayen,et al.  Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control , 2017, ArXiv.

[46]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[47]  Dirk Söffker,et al.  Modeling Driving Behavior at Roundabouts: Impact of Roundabout Layout and Surrounding Traffic on Driving Behavior , 2017 .

[48]  Nadine Rauh,et al.  Facial Expressions as Indicator for Discomfort in Automated Driving , 2020, IHSI.

[49]  Wolfram Burgard,et al.  Self-Supervised Model Adaptation for Multimodal Semantic Segmentation , 2018, International Journal of Computer Vision.

[50]  V. Gallese,et al.  Thermal infrared imaging in psychophysiology: Potentialities and limits , 2014, Psychophysiology.

[51]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Fernando Alonso-Fernandez,et al.  Expression Recognition Using the Periocular Region: A Feasibility Study , 2018, 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[53]  Roberto Cipolla,et al.  Fast-SCNN: Fast Semantic Segmentation Network , 2019, BMVC.

[54]  Stefano Bresciani,et al.  Internet of Things: Applications and challenges in smart cities: a case study of IBM smart city projects , 2016, Bus. Process. Manag. J..

[55]  Jari Saarinen,et al.  Conditional transition maps: Learning motion patterns in dynamic environments , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[56]  Tay Christopher,et al.  Analysis of Dynamic Scenes: Application to Driving Assistance , 2009 .

[57]  M. M. Malone.  European Union, 2020 , 2021 .

[58]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[59]  Raquel Urtasun,et al.  Efficient Convolutions for Real-Time Semantic Segmentation of 3D Point Clouds , 2018, 2018 International Conference on 3D Vision (3DV).

[60]  Daniel Krajzewicz,et al.  Recent Development and Applications of SUMO - Simulation of Urban MObility , 2012 .

[61]  Stefan Byttner,et al.  Consensus self-organized models for fault detection (COSMO) , 2011, Eng. Appl. Artif. Intell..

[62]  Jitendra Malik,et al.  Deformable part models are convolutional neural networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[64]  A. Jazayeri,et al.  Vehicle Detection and Tracking in Car Video Based on Motion Model , 2011, IEEE Transactions on Intelligent Transportation Systems.

[65]  Wei Lu,et al.  Attention Guided Graph Convolutional Networks for Relation Extraction , 2019, ACL.

[66]  Kristoffer Lidström,et al.  On the need for standardized representations of cooperative vehicle behavior , 2013 .

[67]  Arturo de la Escalera,et al.  Sensor Fusion Methodology for Vehicle Detection , 2017, IEEE Intelligent Transportation Systems Magazine.

[68]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[70]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[71]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Lei Chen,et al.  Cooperative ITS — EU standards to accelerate cooperative mobility , 2014, 2014 International Conference on Connected Vehicles and Expo (ICCVE).

[73]  Antanas Verikas,et al.  Ink feed control in a web-fed offset printing press , 2008 .

[74]  Lei Chen,et al.  Cooperative speed harmonization for efficient road utilization , 2014, 2014 7th International Workshop on Communication Technologies for Vehicles (Nets4Cars-Fall).

[75]  Sebastian Scherer,et al.  Season-Invariant Semantic Segmentation with a Deep Multimodal Network , 2017, FSR.

[76]  Albert S. Huang,et al.  A Bayesian nonparametric approach to modeling motion patterns , 2011, Auton. Robots.

[77]  Henk Wymeersch,et al.  Design and Experimental Validation of a Cooperative Driving System in the Grand Cooperative Driving Challenge , 2012, IEEE Transactions on Intelligent Transportation Systems.

[78]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Thomas Brox,et al.  Uncertainty Estimates and Multi-hypotheses Networks for Optical Flow , 2018, ECCV.

[80]  Daniel Krajzewicz,et al.  SUMO (Simulation of Urban MObility) - an open-source traffic simulation , 2002 .

[81]  Wolfram Burgard,et al.  Choosing smartly: Adaptive multimodal fusion for object detection in changing environments , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[82]  D. Mari,et al.  Autonomic Dysfunction in Mild Cognitive Impairment: Evidence from Power Spectral Analysis of Heart Rate Variability in a Cross-Sectional Case-Control Study , 2014, PloS one.

[83]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[84]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  Mohammed Bennamoun,et al.  Deep Learning for 3D Point Clouds: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[86]  Leonidas J. Guibas,et al.  Frustum PointNets for 3D Object Detection from RGB-D Data , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[87]  Tatsuya Harada,et al.  Multispectral Object Detection for Autonomous Vehicles , 2017, ACM Multimedia.

[88]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[89]  T. Gärling,et al.  Travel Demand Management Targeting Reduced Private Car Use: Effectiveness, Public Acceptability and Political Feasibility , 2007 .

[90]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[91]  Jelena Kocić,et al.  Sensors and Sensor Fusion in Autonomous Vehicles , 2018, 2018 26th Telecommunications Forum (TELFOR).

[92]  Hélène Morel,et al.  Body fluid volumes measurements by impedance: A review of bioimpedance spectroscopy (BIS) and bioimpedance analysis (BIA) methods. , 2008, Medical engineering & physics.

[93]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[94]  Bin Yang,et al.  Multi-Task Multi-Sensor Fusion for 3D Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[95]  Min Zhao,et al.  Modeling driver behavior at roundabouts: Results from a field study , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[96]  Cristofer Englund,et al.  Using Recurrent Neural Networks for Action and Intention Recognition of Car Drivers , 2019, ICPRAM.

[97]  Klaus C. J. Dietmayer,et al.  Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges , 2019, IEEE Transactions on Intelligent Transportation Systems.

[98]  Arnold Tukker,et al.  Product services for a resource-efficient and circular economy - A review , 2015 .

[99]  Timo Milbich,et al.  May I enter the roundabout? A time-to-contact computation based on stereo-vision , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[100]  Cristofer Englund.  Aware and Intelligent Infrastructure for Action Intention Recognition of Cars and Bicycles , 2020, VEHITS.

[101]  Kristoffer Lidström,et al.  Act Normal Using Uncertainty about Driver Intentions as a Warning Criterion , 2009 .

[102]  Steven Skiena,et al.  HARP: Hierarchical Representation Learning for Networks , 2017, AAAI.

[103]  C. Qi.  Deep Learning on Point Sets for 3 D Classification and Segmentation , 2016 .

[104]  Fernando Alonso-Fernandez,et al.  A survey on periocular biometrics research , 2016, Pattern Recognit. Lett..

[105]  Daniel Cremers,et al.  FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture , 2016, ACCV.

[106]  Arun Ross,et al.  50 years of biometric research: Accomplishments, challenges, and opportunities , 2016, Pattern Recognit. Lett..

[107]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[108]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[109]  Christian Laugier,et al.  Intention-Aware Risk Estimation for General Traffic Situations, and Application to Intersection Safety , 2013 .

[110]  Max Welling,et al.  Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data , 2020, ArXiv.

[111]  Akhilesh Kumar Maurya,et al.  Acceleration-Deceleration Behaviour of Various Vehicle Types , 2017 .

[112]  Björn Åstrand,et al.  Predicting Agent Behaviour and State for Applications in a Roundabout-Scenario Autonomous Driving , 2019, Sensors.

[113]  Ji Wan,et al.  Multi-view 3D Object Detection Network for Autonomous Driving , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[114]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[115]  Cyrill Stachniss,et al.  RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[116]  Björn Åstrand,et al.  Intention Estimation Using Set of Reference Trajectories as Behaviour Model , 2018, Sensors.

[117]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[118]  Danfei Xu,et al.  PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[119]  Lei Chen,et al.  Cooperative Intersection Management: A Survey , 2016, IEEE Transactions on Intelligent Transportation Systems.

[120]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, ECCV.