A time synchronization protocol for large-scale distributed embedded systems with low-precision clocks and neighbor-to-neighbor communications

Abstract In this paper, we propose the Modular Robot Time Protocol (MRTP), a network-wide time synchronization protocol for modular robots (a class of distributed embedded systems) with neighbor-to-neighbor communications and potentially low-precision clocks. Our protocol achieves its performance by combining several mechanisms: central time master election, fast and recursive propagation of synchronization waves along the edges of a breadth-first spanning-tree, low-level timestamping and per-hop compensation for communication delays using the most-appropriate method, and clock skew compensation using linear regression. We evaluate our protocol on the Blinky Blocks system both on hardware and through simulations. Experimental results show that MRTP can potentially manage real systems composed of up to 27, 775 Blinky Blocks. We observe that the synchronization precision depends on the hop distance to the time master, the synchronization periods and the number of synchronization points used for the linear regressions. Furthermore, we show that our protocol is able to keep a Blinky Blocks system synchronized to a few milliseconds, using few network resources at runtime, even-though the Blinky Blocks hardware clocks exhibit very poor accuracy and resolution. We compare MRTP to existing synchronization protocols ported to fit our system model. Simulation results show that MRTP can achieve better synchronization precision than the most precise compared protocols while sending more than half less messages in compact systems.

[1]  Seth Copen Goldstein,et al.  Blinky blocks: a physical ensemble programming platform , 2011, CHI EA '11.

[2]  Nitin H. Vaidya,et al.  Leader election algorithms for mobile ad hoc networks , 2000, DIALM '00.

[3]  Yik-Chung Wu,et al.  On Clock Synchronization Algorithms for Wireless Sensor Networks Under Unknown Delay , 2010, IEEE Transactions on Vehicular Technology.

[4]  Julien Bourgeois,et al.  Distributed Intelligent MEMS: Progresses and Perspectives , 2011, IEEE Systems Journal.

[5]  Hicham Lakhlef,et al.  Programmable matter as a cyber-physical conjugation , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[6]  Jiming Chen,et al.  Study of consensus-based time synchronization in wireless sensor networks. , 2014, ISA transactions.

[7]  Benjamin R. Hamilton,et al.  Tracking Low-Precision Clocks With Time-Varying Drifts Using Kalman Filtering , 2012, IEEE/ACM Transactions on Networking.

[8]  Flaviu Cristian,et al.  Probabilistic clock synchronization , 1989, Distributed Computing.

[9]  Gyula Simon,et al.  The flooding time synchronization protocol , 2004, SenSys '04.

[10]  Branislav Kusy,et al.  Spatiotemporal coordination in wireless sensor networks , 2007 .

[11]  Donald F. Towsley,et al.  Design and analysis of a leader election algorithm for mobile ad hoc networks , 2004, Proceedings of the 12th IEEE International Conference on Network Protocols, 2004. ICNP 2004..

[12]  D. W. Allan,et al.  Time and Frequency (Time-Domain) Characterization, Estimation, and Prediction of Precision Clocks and Oscillators , 1987, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[13]  Bruce W. Suter,et al.  Novel Clock Phase Offset and Skew Estimation Using Two-Way Timing Message Exchanges for Wireless Sensor Networks , 2007, IEEE Transactions on Communications.

[14]  Rafael Aracil,et al.  Synchronizing a Modular Robot Colony for Cooperative Tasks Based on Intra-Inter Robot Communications , 2010, 2010 IEEE Electronics, Robotics and Automotive Mechanics Conference.

[15]  Satoshi Murata,et al.  Clock Synchronization Mechanisms for a Distributed Autonomous System , 1996, J. Robotics Mechatronics.

[16]  Christoph Lenzen,et al.  Optimal clock synchronization in networks , 2009, SenSys '09.

[17]  Ling Shi,et al.  Time Synchronization in WSNs: A Maximum-Value-Based Consensus Approach , 2014, IEEE Transactions on Automatic Control.

[18]  Saurabh Ganeriwal,et al.  Timing-sync protocol for sensor networks , 2003, SenSys '03.

[19]  Luca Schenato,et al.  Average TimeSynch: A consensus-based protocol for clock synchronization in wireless sensor networks , 2011, Autom..

[20]  Roger Wattenhofer,et al.  Gradient clock synchronization in wireless sensor networks , 2009, 2009 International Conference on Information Processing in Sensor Networks.

[21]  Lothar Thiele,et al.  Efficient network flooding and time synchronization with Glossy , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[22]  Kay Römer,et al.  Time Synchronization and Calibration in Wireless Sensor Networks , 2005, Handbook of Sensor Networks.

[23]  Erol Sahin,et al.  Swarm Robotics: From Sources of Inspiration to Domains of Application , 2004, Swarm Robotics.

[24]  Julien Bourgeois,et al.  A Distributed Algorithm for Reconfiguration of Lattice-Based Modular Self-Reconfigurable Robots , 2016, 2016 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP).

[25]  P. Flajolet,et al.  HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm , 2007 .

[26]  Nikolaus Correll,et al.  Materials that couple sensing, actuation, computation, and communication , 2015, Science.

[27]  Roger Wattenhofer,et al.  Slotted programming for sensor networks , 2010, IPSN '10.

[28]  Christoph Lenzen,et al.  PulseSync: An Efficient and Scalable Clock Synchronization Protocol , 2015, IEEE/ACM Transactions on Networking.

[29]  Kang Lee,et al.  IEEE 1588 standard for a precision clock synchronization protocol for networked measurement and control systems , 2002, 2nd ISA/IEEE Sensors for Industry Conference,.

[30]  Mary Wu,et al.  Leader election on tree-based centrality in ad hoc networks , 2013, Telecommun. Syst..

[31]  Henk Wymeersch,et al.  Cooperative Synchronization in Wireless Networks , 2013, IEEE Transactions on Signal Processing.

[32]  Seth Copen Goldstein,et al.  Claytronics: An Instance of Programmable Matter , 2004 .

[33]  Nikolaus Correll,et al.  Bloom Filter-Based Ad Hoc Multicast Communication in Cyber-Physical Systems and Computational Materials , 2012, WASA.

[34]  Xenofon D. Koutsoukos,et al.  Time Synchronization in Heterogeneous Sensor Networks , 2008, DCOSS.

[35]  Mark Yim,et al.  Modular Self-Reconfigurable Robots , 2009, Encyclopedia of Complexity and Systems Science.

[36]  Julien Bourgeois,et al.  ABC-Center: Approximate-center election in modular robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[37]  Julien Bourgeois,et al.  Distributed Pneumatic MEMS for Fast Conveyance of Fragile Objects , 2012 .

[38]  Hicham Lakhlef,et al.  Optimization of the logical topology for mobile MEMS networks , 2014, J. Netw. Comput. Appl..

[39]  Julien Bourgeois,et al.  Approximate-Centroid Election in Large-Scale Distributed Embedded Systems , 2016, 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA).

[40]  Deborah Estrin,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Fine-grained Network Time Synchronization Using Reference Broadcasts , 2022 .

[41]  Julien Bourgeois,et al.  A Time Synchronization Protocol for Modular Robots , 2016, 2016 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP).

[42]  Julien Bourgeois,et al.  Efficient Simulation of Distributed Sensing and Control Environments , 2013, 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing.