Coding-based system primitives for airborne cloud computing

The recent proliferation of sensors in inhospitable environments such as disaster or battle zones has not been matched by in situ data processing capabilities due to a lack of computing infrastructure in the field. We envision a solution based on small, low-altitude unmanned aerial vehicles (UAVs) that can deploy elastically-scalable computing infrastructure anywhere, at any time. This airborne compute cloud—essentially, micro-data centers hosted on UAVs—would communicate with terrestrial assets over a bandwidth-constrained wireless network with variable, unpredictable link qualities. Achieving high performance over this ground-to-air mobile radio channel thus requires making full and efficient use of every single transmission opportunity. To this end, this dissertation presents two system primitives that improve throughput and reduce network overhead by using recent distributed coding methods to exploit natural properties of the airborne environment (i.e., antenna beam diversity and anomaly sparsity). We first built and deployed an UAV wireless networking testbed and used it to characterize the ground-to-UAV wireless channel. Our flight experiments revealed that antenna beam diversity from using multiple SISO radios boosts reception range and aggregate throughput. This observation led us to develop our first primitive: ground-to-UAV bulk data transport. We designed and implemented FlowCode, a reliable link layer for uplink data transport that uses network coding to harness antenna beam diversity gains. Via flight experiments, we show that FlowCode can boost reception range and TCP throughput as much as 4.5-fold. Our second primitive permits low-overhead cloud status monitoring. We designed CloudSense, a network switch that compresses cloud status streams in-network via compressive sensing. CloudSense is particularly useful for anomaly detection tasks requiring global relative comparisons (e.g., MapReduce straggler detection) and can achieve up to 16.3-fold compression as well as early detection of the worst anomalies. Our efforts have also shed light on the close relationship between network coding and compressive sensing. Thus, we offer FlowCode and CloudSense not only as first steps toward the airborne compute cloud, but also as exemplars of two classes of applications—approximation intolerant and tolerant—to which network coding and compressive sensing should be judiciously and selectively applied.

[1]  Injong Rhee,et al.  CUBIC: a new TCP-friendly high-speed TCP variant , 2008, OPSR.

[2]  Bruce W. Suter,et al.  Measurement combining and progressive reconstruction in compressive sensing , 2011, 2011 - MILCOM 2011 Military Communications Conference.

[3]  R. Koetter,et al.  The benefits of coding over routing in a randomized setting , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[4]  Dina Katabi,et al.  SourceSync: a distributed wireless architecture for exploiting sender diversity , 2010, SIGCOMM '10.

[5]  Jörg Widmer,et al.  Network coding for efficient communication in extreme networks , 2005, WDTN '05.

[6]  Yong Yao,et al.  The cougar approach to in-network query processing in sensor networks , 2002, SGMD.

[7]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[8]  Thomas Hofmann,et al.  Map-Reduce for Machine Learning on Multicore , 2007 .

[9]  Abhinandan Das,et al.  Google news personalization: scalable online collaborative filtering , 2007, WWW '07.

[10]  Devavrat Shah,et al.  Network Coding Meets TCP , 2008, IEEE INFOCOM 2009.

[11]  Deborah Estrin,et al.  The impact of data aggregation in wireless sensor networks , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[12]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[13]  Thierry Turletti,et al.  Network coding for wireless mesh networks: a case study , 2006, 2006 International Symposium on a World of Wireless, Mobile and Multimedia Networks(WoWMoM'06).

[14]  Vishnu Navda,et al.  A measurement study of inter-vehicular communication using steerable beam directional antenna , 2008, VANET '08.

[15]  D. Lun,et al.  Methods for Efficient Network Coding , 2006 .

[16]  Jon Postel,et al.  Transmission Control Protocol , 1981, RFC.

[17]  Michael Isard,et al.  Autopilot: automatic data center management , 2007, OPSR.

[18]  Dario Vlah,et al.  Speculative pipelining for compute cloud programming , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[19]  Kannan Ramchandran,et al.  Distributed compression in a dense microsensor network , 2002, IEEE Signal Process. Mag..

[20]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[21]  Geoffrey C. Fox,et al.  MapReduce for Data Intensive Scientific Analyses , 2008, 2008 IEEE Fourth International Conference on eScience.

[22]  Glen Gibb,et al.  NetFPGA--An Open Platform for Gigabit-Rate Network Switching and Routing , 2007, 2007 IEEE International Conference on Microelectronic Systems Education (MSE'07).

[23]  Emin Gün Sirer,et al.  SideCar: building programmable datacenter networks without programmable switches , 2010, Hotnets-IX.

[24]  R. Koetter,et al.  An algebraic approach to network coding , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[25]  Liviu Iftode,et al.  Improving the Performance of Reliable Transport Protocols in Mobile Computing Environments , 1994, IEEE J. Sel. Areas Commun..

[26]  Mircea Andrecut,et al.  Fast GPU Implementation of Sparse Signal Recovery from Random Projections , 2008, Eng. Lett..

[27]  H. Vincent Poor,et al.  Distributed transmit beamforming: challenges and recent progress , 2009, IEEE Communications Magazine.

[28]  Stephen J. Wright,et al.  Implementing Algorithms for Signal and Image Reconstruction on Graphical Processing Units , 2008 .

[29]  Vasaka Visoottiviseth,et al.  An empirical study on achievable throughputs of IEEE 802.11n devices , 2009, 2009 7th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks.

[30]  Feng Jiang,et al.  Dynamic UAV relay positioning for the ground-to-air uplink , 2010, 2010 IEEE Globecom Workshops.

[31]  Kamran Mohseni,et al.  SensorFlock: an airborne wireless sensor network of micro-air vehicles , 2007, SenSys '07.

[32]  Sachin Katti,et al.  Trading structure for randomness in wireless opportunistic routing , 2007, SIGCOMM 2007.

[33]  Andreas Timm-Giel,et al.  MobiSteer: using steerable beam directional antenna for vehicular network access , 2007, MobiSys '07.

[34]  R. D'Andrea,et al.  Using airborne vehicle-based antenna arrays to improve communications with UAV clusters , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[35]  R. Baraniuk,et al.  Compressive Radar Imaging , 2007, 2007 IEEE Radar Conference.

[36]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[37]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[38]  Theodore S. Rappaport,et al.  Wireless Communications: Principles and Practice (2nd Edition) by , 2012 .

[39]  R. A. McDonald,et al.  Noiseless Coding of Correlated Information Sources , 1973 .

[40]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[41]  Muriel Médard,et al.  Codecast: a network-coding-based ad hoc multicast protocol , 2006, IEEE Wireless Communications.

[42]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[43]  Vishnu Navda,et al.  Efficient gathering of correlated data in sensor networks , 2008, TOSN.

[44]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[45]  Dario Vlah,et al.  A location-dependent runs-and-gaps model for predicting TCP performance over a UAV wireless channel , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[46]  Robert Morris,et al.  Link-level measurements from an 802.11b mesh network , 2004, SIGCOMM 2004.

[47]  K. Leentvaar,et al.  The Capture Effect in FM Receivers , 1976, IEEE Trans. Commun..

[48]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[49]  Martin Vetterli,et al.  Network correlated data gathering with explicit communication: NP-completeness and algorithms , 2006 .

[50]  Harald Niederreiter,et al.  Probability and computing: randomized algorithms and probabilistic analysis , 2006, Math. Comput..

[51]  H. T. Kung,et al.  Transmit Antenna Selection Based on Link-layer Channel Probing , 2007, 2007 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[52]  Jun Sun,et al.  Compressive data gathering for large-scale wireless sensor networks , 2009, MobiCom '09.

[53]  Jérôme Darbon,et al.  A Simple Compressive Sensing Algorithm for Parallel Many-Core Architectures , 2013, J. Signal Process. Syst..

[54]  Douglas L. Jones,et al.  Netcompress: Coupling network coding and compressed sensing for efficient data communication in wireless sensor networks , 2010, 2010 IEEE Workshop On Signal Processing Systems.

[55]  Yanghee Choi,et al.  An experimental study on the capture effect in 802.11a networks , 2007, WinTECH '07.

[56]  Baltasar Beferull-Lozano,et al.  On network correlated data gathering , 2004, IEEE INFOCOM 2004.

[57]  Tarek F. Abdelzaher,et al.  AIDA: Adaptive application-independent data aggregation in wireless sensor networks , 2004, TECS.

[58]  Divyakant Agrawal,et al.  Medians and beyond: new aggregation techniques for sensor networks , 2004, SenSys '04.

[59]  Albert G. Greenberg,et al.  Reining in the Outliers in Map-Reduce Clusters using Mantri , 2010, OSDI.

[60]  Timothy X. Brown,et al.  Ad Hoc UAV Ground Network (AUGNet) , 2004 .

[61]  Muriel Medard,et al.  On Randomized Network Coding , 2003 .

[62]  H.T. Kung,et al.  Field Experimentation of Cots-Based UAV Networking , 2006, MILCOM 2006 - 2006 IEEE Military Communications conference.

[63]  Muriel Médard,et al.  Algebraic gossip: a network coding approach to optimal multiple rumor mongering , 2006, IEEE Transactions on Information Theory.

[64]  H.T. Kung,et al.  A computational wireless network backplane: Performance in a distributed speaker identification application , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[65]  Y. Rachlin,et al.  The secrecy of compressed sensing measurements , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[66]  Bruno Clerckx,et al.  MIMO techniques in WiMAX and LTE: a feature overview , 2010, IEEE Communications Magazine.

[67]  Muriel Médard,et al.  XORs in the Air: Practical Wireless Network Coding , 2006, IEEE/ACM Transactions on Networking.

[68]  P. Kumar,et al.  Capacity of Ad Hoc Wireless Networks , 2002 .

[69]  K. Jain,et al.  Practical Network Coding , 2003 .

[70]  Federico D. Sacerdoti,et al.  Scalable Algorithms for Molecular Dynamics Simulations on Commodity Clusters , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[71]  Wei Hong,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Tag: a Tiny Aggregation Service for Ad-hoc Sensor Networks , 2022 .

[72]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[73]  A. Lee Swindlehurst,et al.  Wireless Relay Communications with Unmanned Aerial Vehicles: Performance and Optimization , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[74]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[75]  Konstantinos Pelechrinis,et al.  Experimental characterization of 802.11n link quality at high rates , 2010, WiNTECH '10.

[76]  H. T. Kung,et al.  Maximizing Throughput of UAV-Relaying Networks with the Load-Carry-and-Deliver Paradigm , 2007, 2007 IEEE Wireless Communications and Networking Conference.

[77]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[78]  Jean C. Walrand,et al.  Analysis and comparison of TCP Reno and Vegas , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[79]  R.C. Palat,et al.  Cooperative relaying for ad-hoc ground networks using swarm UAVs , 2005, MILCOM 2005 - 2005 IEEE Military Communications Conference.

[80]  GhemawatSanjay,et al.  The Google file system , 2003 .

[81]  A. Stephens,et al.  Wi-Fi (802.11b) and Bluetooth: enabling coexistence , 2001, IEEE Netw..

[82]  J. D. Parsons,et al.  The Mobile Radio Propagation Channel , 1991 .

[83]  Lawrence G. Roberts,et al.  ALOHA packet system with and without slots and capture , 1975, CCRV.

[84]  Kannan Ramchandran,et al.  A distributed and adaptive signal processing approach to reducing energy consumption in sensor networks , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[85]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[86]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[87]  H. T. Kung,et al.  Performance Measurement of 802.11a Wireless Links from UAV to Ground Nodes with Various Antenna Orientations , 2006, Proceedings of 15th International Conference on Computer Communications and Networks.

[88]  Albert G. Greenberg,et al.  The cost of a cloud: research problems in data center networks , 2008, CCRV.

[89]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[90]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[91]  Giovanni Pau,et al.  Code torrent: content distribution using network coding in VANET , 2006, MobiShare '06.

[92]  Ting Liu,et al.  Clustering Billions of Images with Large Scale Nearest Neighbor Search , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[93]  Rudolf Ahlswede,et al.  Network information flow , 2000, IEEE Trans. Inf. Theory.

[94]  Shuo-Yen Robert Li,et al.  Linear network coding , 2003, IEEE Trans. Inf. Theory.

[95]  Robert W. Heath,et al.  Shifting the MIMO Paradigm , 2007, IEEE Signal Processing Magazine.

[96]  Sampath Rangarajan,et al.  R2D2: regulating beam shape and rate as directionality meets diversity , 2009, MobiSys '09.

[97]  Haitao Wu,et al.  ServerSwitch: A Programmable and High Performance Platform for Data Center Networks , 2011, NSDI.

[98]  Richard G Baraniuk,et al.  More Is Less: Signal Processing and the Data Deluge , 2011, Science.

[99]  Michael Mitzenmacher,et al.  Digital fountains: a survey and look forward , 2004, Information Theory Workshop.

[100]  Yunnan Wu,et al.  Information Exchange in Wireless Networks with Network Coding and Physical-layer Broadcast , 2004 .

[101]  Dario Vlah,et al.  Antenna selection performance in 802.11 networks , 2007 .

[102]  H. T. Kung,et al.  Separation-Based Joint Decoding in Compressive Sensing , 2011, 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN).

[103]  Ashok K. Agrawala,et al.  Sniffing out the correct physical layer capture model in 802.11b , 2004, Proceedings of the 12th IEEE International Conference on Network Protocols, 2004. ICNP 2004..

[104]  Kurt Keutzer,et al.  Fast support vector machine training and classification on graphics processors , 2008, ICML '08.

[105]  Chuan Qin,et al.  I²MIX: Integration of Intra-Flow and Inter-Flow Wireless Network Coding , 2008, 2008 5th IEEE Annual Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks Workshops.