Collaborative Autonomous Driving: Vision and Challenges

This paper discusses challenges in computer systems research posed by the emerging autonomous driving systems. We first identify four research areas related to autonomous driving systems: real-time and embedded systems, machine learning, edge computing, and cloud computing. Next, we sketch two fatal accidents caused by active autonomous driving, and uses them to indicate key missing capabilities from today’s systems. In light of these research areas and shortcomings, we describe a vision of digital driving circumstances for autonomous vehicles and refer to autonomous vehicles as "clients" of this digital driving circumstance. Then we propose a new research thrust: collaborative autonomous driving. Intuitively, requesting useful information from a digital driving circumstance to enable collaborative autonomous driving is quite sophisticated (e.g., collaborations may come from different types of unstable edge devices), but it also provide us various research challenges and opportunities. The paper closes with a discussion of the research necessary to develop these capabilities.

[1]  Pushmeet Kohli,et al.  Associative Hierarchical Random Fields , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ming Yang,et al.  Reducing Response-Time Bounds for DAG-Based Task Systems on Heterogeneous Multicore Platforms , 2016, RTNS.

[3]  Insup Lee,et al.  Hierarchical Scheduling Framework for Virtual Clustering of Multiprocessors , 2008, 2008 Euromicro Conference on Real-Time Systems.

[4]  Jonathan Walpole,et al.  Analysis of a reservation-based feedback scheduler , 2002, 23rd IEEE Real-Time Systems Symposium, 2002. RTSS 2002..

[5]  Theodore P. Baker,et al.  Stack-based scheduling of realtime processes , 1991, Real-Time Systems.

[6]  Shinpei Kato,et al.  Gdev: First-Class GPU Resource Management in the Operating System , 2012, USENIX Annual Technical Conference.

[7]  Shinpei Kato,et al.  RGEM: A Responsive GPGPU Execution Model for Runtime Engines , 2011, 2011 IEEE 32nd Real-Time Systems Symposium.

[8]  James H. Anderson,et al.  The OMLP family of optimal multiprocessor real-time locking protocols , 2013, Des. Autom. Embed. Syst..

[9]  Steve Vestal,et al.  Preemptive Scheduling of Multi-criticality Systems with Varying Degrees of Execution Time Assurance , 2007, 28th IEEE International Real-Time Systems Symposium (RTSS 2007).

[10]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  James H. Anderson,et al.  OpenVX and Real-Time Certification: The Troublesome History , 2019, 2019 IEEE Real-Time Systems Symposium (RTSS).

[12]  James H. Anderson,et al.  Fine-grained task reweighting on multiprocessors , 2005, 11th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA'05).

[13]  Ming Yang,et al.  GPU Scheduling on the NVIDIA TX2: Hidden Details Revealed , 2017, 2017 IEEE Real-Time Systems Symposium (RTSS).

[14]  Zheng Dong,et al.  Closing the Loop for the Selective Conversion Approach: A Utilization-Based Test for Hard Real-Time Suspending Task Systems , 2016, 2016 IEEE Real-Time Systems Symposium (RTSS).

[15]  Lui Sha,et al.  Real-time synchronization protocols for multiprocessors , 1988, Proceedings. Real-Time Systems Symposium.

[16]  Andreas Geiger,et al.  Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art , 2017, Found. Trends Comput. Graph. Vis..

[17]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Silvio Savarese,et al.  Ieee Transaction on Pattern Analysis and Machine Intelligence 1 a General Framework for Tracking Multiple People from a Moving Camera , 2022 .

[19]  Weisong Shi,et al.  Edge Computing for Autonomous Driving: Opportunities and Challenges , 2019, Proceedings of the IEEE.

[20]  Lui Sha,et al.  Priority Inheritance Protocols: An Approach to Real-Time Synchronization , 1990, IEEE Trans. Computers.

[21]  James H. Anderson,et al.  Exploring the Multitude of Real-Time Multi-GPU Configurations , 2014, 2014 IEEE Real-Time Systems Symposium.

[22]  James H. Anderson,et al.  Supporting Real-Time Computer Vision Workloads Using OpenVX on Multicore+GPU Platforms , 2015, 2015 IEEE Real-Time Systems Symposium.

[23]  George C. Zalidis,et al.  An autonomous multi-sensor UAV system for reduced-input precision agriculture applications , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[24]  Xiaolong Hu,et al.  Autonomous Driving in the iCity—HD Maps as a Key Challenge of the Automotive Industry , 2016 .

[25]  Hussein M. Abdel-Wahab,et al.  A proportional share resource allocation algorithm for real-time, time-shared systems , 1996, 17th IEEE Real-Time Systems Symposium.

[26]  Pushmeet Kohli,et al.  Graph Cut Based Inference with Co-occurrence Statistics , 2010, ECCV.

[27]  James H. Anderson,et al.  Globally scheduled real-time multiprocessor systems with GPUs , 2011, Real-Time Systems.

[28]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Hennadiy Leontyev,et al.  A Flexible Real-Time Locking Protocol for Multiprocessors , 2007, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007).

[30]  Sanjoy Baruah Rapid Routing with Guaranteed Delay Bounds , 2018, 2018 IEEE Real-Time Systems Symposium (RTSS).

[31]  Yanhua Li,et al.  REC: Predictable Charging Scheduling for Electric Taxi Fleets , 2017, 2017 IEEE Real-Time Systems Symposium (RTSS).

[32]  Hennadiy Leontyev,et al.  A hierarchical multiprocessor bandwidth reservation scheme with timing guarantees , 2008, 2008 Euromicro Conference on Real-Time Systems.

[33]  Sanjoy K. Baruah,et al.  Virtual Multiprocessor Platforms: Specification and Use , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[34]  Avi Mendelson,et al.  Batch Method for Efficient Resource Sharing in Real-Time Multi-GPU Systems , 2014, ICDCN.

[35]  Jane W.-S. Liu,et al.  Scheduling real-time applications in an open environment , 1997, Proceedings Real-Time Systems Symposium.

[36]  Liang He,et al.  Exploring Adaptive Reconfiguration to Optimize Energy Efficiency in Large-Scale Battery Systems , 2013, 2013 IEEE 34th Real-Time Systems Symposium.

[37]  Zheng Dong,et al.  Analysis Techniques for Supporting Hard Real-Time Sporadic Gang Task Systems , 2017, 2017 IEEE Real-Time Systems Symposium (RTSS).

[38]  Insup Lee,et al.  Cache-aware compositional analysis of real-time multicore virtualization platforms , 2013, 2013 IEEE 34th Real-Time Systems Symposium.

[39]  Kwei-Jay Lin,et al.  Dynamic priority ceilings: A concurrency control protocol for real-time systems , 1990, Real-Time Systems.

[40]  B. Bett,et al.  Autonomous Underwater Vehicles (AUVs): Their past, present and future contributions to the advancement of marine geoscience , 2014 .

[41]  Insup Lee,et al.  Compositional Analysis Framework Using EDP Resource Models , 2007, 28th IEEE International Real-Time Systems Symposium (RTSS 2007).

[42]  Konrad Schindler,et al.  Detection- and Trajectory-Level Exclusion in Multiple Object Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  James H. Anderson,et al.  Multi-resource Real-Time Reader/Writer Locks for Multiprocessors , 2014, 2014 IEEE 28th International Parallel and Distributed Processing Symposium.

[44]  Liang He,et al.  A General Analysis Framework for Soft Real-Time Tasks , 2019, IEEE Transactions on Parallel and Distributed Systems.

[45]  Kyoung-Don Kang,et al.  Supporting Preemptive Task Executions and Memory Copies in GPGPUs , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[46]  Sang Hyuk Son,et al.  Performance specifications and metrics for adaptive real-time systems , 2000, Proceedings 21st IEEE Real-Time Systems Symposium.

[47]  Giuseppe Lipari,et al.  A Framework for Hierarchical Scheduling on Multiprocessors: From Application Requirements to Run-Time Allocation , 2010, 2010 31st IEEE Real-Time Systems Symposium.

[48]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Insup Lee,et al.  Periodic resource model for compositional real-time guarantees , 2003, RTSS 2003. 24th IEEE Real-Time Systems Symposium, 2003.

[50]  James H. Anderson,et al.  GPUSync: A Framework for Real-Time GPU Management , 2013, 2013 IEEE 34th Real-Time Systems Symposium.

[51]  David G. Schmale,et al.  Coordinated aerobiological sampling of a plant pathogen in the lower atmosphere using two autonomous unmanned aerial vehicles , 2010, J. Field Robotics.

[52]  Ming Yang,et al.  Avoiding Pitfalls when Using NVIDIA GPUs for Real-Time Tasks in Autonomous Systems , 2018, ECRTS.

[53]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[54]  Ming Yang,et al.  Making OpenVX Really "Real Time" , 2018, 2018 IEEE Real-Time Systems Symposium (RTSS).

[55]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[56]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  James H. Anderson,et al.  Optimality Results for Multiprocessor Real-Time Locking , 2010, 2010 31st IEEE Real-Time Systems Symposium.

[58]  James H. Anderson,et al.  Supporting Nested Locking in Multiprocessor Real-Time Systems , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[59]  Giuseppe Lipari,et al.  A comparison of MPCP and MSRP when sharing resources in the Janus multiple-processor on a chip platform , 2003, The 9th IEEE Real-Time and Embedded Technology and Applications Symposium, 2003. Proceedings..

[60]  Insup Lee,et al.  Optimal virtual cluster-based multiprocessor scheduling , 2009, Real-Time Systems.

[61]  R. Rajkumar Real-time synchronization protocols for shared memory multiprocessors , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[62]  James H. Anderson,et al.  Optimal Dataflow Scheduling on a Heterogeneous Multiprocessor With Reduced Response Time Bounds , 2017, ECRTS.

[63]  Shinpei Kato,et al.  TimeGraph: GPU Scheduling for Real-Time Multi-Tasking Environments , 2011, USENIX Annual Technical Conference.

[64]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[65]  Robert I. Davis,et al.  Mixed Criticality Systems - A Review , 2015 .

[66]  Konrad Schindler,et al.  Multi-target tracking by continuous energy minimization , 2011, CVPR 2011.

[67]  Zheng Dong,et al.  An energy-efficient offloading framework with predictable temporal correctness , 2017, SEC.

[68]  Weisong Shi,et al.  Equinox: A Road-Side Edge Computing Experimental Platform for CAVs , 2020, 2020 International Conference on Connected and Autonomous Driving (MetroCAD).

[69]  Giorgio C. Buttazzo,et al.  The Multi Supply Function Abstraction for Multiprocessors , 2009, 2009 15th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications.

[70]  Weisong Shi,et al.  E2M: an energy-efficient middleware for computer vision applications on autonomous mobile robots , 2019, SEC.

[71]  Sang Hyuk Son,et al.  Feedback Control Real-Time Scheduling: Framework, Modeling, and Algorithms* , 2001, Real-Time Systems.

[72]  James H. Anderson,et al.  Robust Real-Time Multiprocessor Interrupt Handling Motivated by GPUs , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[73]  Giuseppe Lipari,et al.  Resource partitioning among real-time applications , 2003, 15th Euromicro Conference on Real-Time Systems, 2003. Proceedings..

[74]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[75]  Cong Liu,et al.  GPES: a preemptive execution system for GPGPU computing , 2015, 21st IEEE Real-Time and Embedded Technology and Applications Symposium.

[76]  Wen Zhang,et al.  Tracking Hit-and-Run Vehicle with Sparse Video Surveillance Cameras and Mobile Taxicabs , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[77]  Avi Mendelson,et al.  Scheduling processing of real-time data streams on heterogeneous multi-GPU systems , 2012, SYSTOR '12.

[78]  James H. Anderson,et al.  Fine-grained multiprocessor real-time locking with improved blocking , 2013, RTNS '13.