Cognition-Based Networks: A New Perspective on Network Optimization Using Learning and Distributed Intelligence

In response to the new challenges in the design and operation of communication networks, and taking inspiration from how living beings deal with complexity and scalability, in this paper we introduce an innovative system concept called COgnition-BAsed NETworkS (COBANETS). The proposed approach develops around the systematic application of advanced machine learning techniques and, in particular, unsupervised deep learning and probabilistic generative models for system-wide learning, modeling, optimization, and data representation. Moreover, in COBANETS, we propose to combine this learning architecture with the emerging network virtualization paradigms, which make it possible to actuate automatic optimization and reconfiguration strategies at the system level, thus fully unleashing the potential of the learning approach. Compared with the past and current research efforts in this area, the technical approach outlined in this paper is deeply interdisciplinary and more comprehensive, calling for the synergic combination of expertise of computer scientists, communications and networking engineers, and cognitive scientists, with the ultimate aim of breaking new ground through a profound rethinking of how the modern understanding of cognition can be used in the management and optimization of telecommunication networks.

[1]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[2]  Hari Balakrishnan,et al.  TCP ex machina: computer-generated congestion control , 2013, SIGCOMM.

[3]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[4]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[5]  Ian F. Akyildiz,et al.  CRAHNs: Cognitive radio ad hoc networks , 2009, Ad Hoc Networks.

[6]  Mischa Dohler,et al.  Docitive networks: an emerging paradigm for dynamic spectrum management [Dynamic Spectrum Management] , 2010, IEEE Wireless Communications.

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  V. Tarokh,et al.  Cognitive radio networks , 2008, IEEE Signal Processing Magazine.

[9]  Sebastian Zander,et al.  A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification , 2006, CCRV.

[10]  Yoshua Bengio,et al.  Justifying and Generalizing Contrastive Divergence , 2009, Neural Computation.

[11]  Nino Vincenzo Verde,et al.  Can't You Hear Me Knocking: Identification of User Actions on Android Apps via Traffic Analysis , 2014, CODASPY.

[12]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[13]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[14]  Panganamala Ramana Kumar,et al.  A cautionary perspective on cross-layer design , 2005, IEEE Wireless Communications.

[15]  Anirban Mahanti,et al.  Traffic classification using clustering algorithms , 2006, MineNet '06.

[16]  Carolina Fortuna,et al.  Trends in the development of communication networks: Cognitive networks , 2009, Comput. Networks.

[17]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[18]  Marco Zorzi,et al.  Parallelization of Deep Networks , 2012, ESANN.

[19]  Shimon Whiteson,et al.  Adaptive job routing and scheduling , 2004, Eng. Appl. Artif. Intell..

[20]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[21]  Karl J. Friston,et al.  Perceptions as Hypotheses: Saccades as Experiments , 2012, Front. Psychology.

[22]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[23]  Jim Esch,et al.  Software-Defined Networking: A Comprehensive Survey , 2015, Proc. IEEE.

[24]  Marco Zorzi,et al.  Emergence of a 'visual number sense' in hierarchical generative models , 2012, Nature Neuroscience.

[25]  A. Robert Calderbank,et al.  Content-Aware Distortion-Fair Video Streaming in Congested Networks , 2009, IEEE Transactions on Multimedia.

[26]  David D. Clark,et al.  A knowledge plane for the internet , 2003, SIGCOMM '03.

[27]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[28]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  M. Motani,et al.  Cross-layer design: a survey and the road ahead , 2005, IEEE Communications Magazine.

[30]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[31]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[32]  Rajat Raina,et al.  Large-scale deep unsupervised learning using graphics processors , 2009, ICML '09.

[33]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[34]  Shilpa Achaliya,et al.  Cognitive radio , 2010 .

[35]  Kevin Skadron,et al.  Scalable parallel programming , 2008, 2008 IEEE Hot Chips 20 Symposium (HCS).

[36]  Zwi Altman,et al.  A cooperative Reinforcement Learning approach for Inter-Cell Interference Coordination in OFDMA cellular networks , 2010, 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks.

[37]  Sudharman K. Jayaweera,et al.  A Survey on Machine-Learning Techniques in Cognitive Radios , 2013, IEEE Communications Surveys & Tutorials.

[38]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[39]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[40]  Andreas Mitschele-Thiel,et al.  Reinforcement learning strategies for self-organized coverage and capacity optimization , 2012, 2012 IEEE Wireless Communications and Networking Conference (WCNC).

[41]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[42]  Rouzbeh Razavi,et al.  Self-optimization of capacity and coverage in LTE networks using a fuzzy reinforcement learning approach , 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[43]  Oliver Hohlfeld,et al.  Impact of frame rate and resolution on objective QoE metrics , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[44]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[45]  Mauro Conti,et al.  Cache Privacy in Named-Data Networking , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[46]  Timothy J. O'Shea,et al.  Applications of Machine Learning to Cognitive Radio Networks , 2007, IEEE Wireless Communications.

[47]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[48]  Ling Huang,et al.  Adversarial Active Learning , 2014, AISec '14.

[49]  Wolfgang Kellerer,et al.  QoE-Based Cross-Layer Optimization of Wireless Video with Unperceivable Temporal Video Quality Fluctuation , 2011, 2011 IEEE International Conference on Communications (ICC).

[50]  Andrew W. Moore,et al.  Internet traffic classification using bayesian analysis techniques , 2005, SIGMETRICS '05.

[51]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[52]  Alessandro Sperduti,et al.  Learning Orthographic Structure With Sequential Generative Neural Networks , 2016, Cogn. Sci..

[53]  Huseyin Arslan,et al.  Cognitive radio, software defined radio, and adaptiv wireless systems , 2007 .

[54]  Joseph Mitola An Integrated Agent Architecture for Software Defined Radio , 2000 .

[55]  Srinivasan Seshan,et al.  Developing a predictive model of quality of experience for internet video , 2013, SIGCOMM.

[56]  Geoffrey E. Hinton,et al.  Unsupervised learning : foundations of neural computation , 1999 .

[57]  Michele De Filippo De Grazia,et al.  Deep Unsupervised Learning on a Desktop PC: A Primer for Cognitive Scientists , 2013, Front. Psychol..

[58]  Michele Zorzi,et al.  CARMEN: a cognitive networking testbed on android OS devices , 2014, IEEE Communications Magazine.

[59]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[60]  Fernando M. V. Ramos,et al.  Software-Defined Networking: A Comprehensive Survey This paper offers a comprehensive survey of software-defined networking covering its context, rationale, main concepts, distinctive features, and future challenges. , 2015 .

[61]  Muhammad Ali Imran,et al.  A Survey of Self Organisation in Future Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[62]  Laura Dal Col,et al.  A linear consensus approach to quality-fair video delivery , 2014, 53rd IEEE Conference on Decision and Control.

[63]  B. Frey,et al.  The human splicing code reveals new insights into the genetic determinants of disease , 2015, Science.

[64]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[65]  Geoffrey E. Hinton,et al.  Factored conditional restricted Boltzmann Machines for modeling motion style , 2009, ICML '09.

[66]  Leonardo Badia,et al.  Cognition-based networks: Applying cognitive science to multimedia wireless networking , 2014, Proceeding of IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks 2014.

[67]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.

[68]  Alberto Testolin,et al.  Modeling language and cognition with deep unsupervised learning: a tutorial overview , 2013, Front. Psychol..

[69]  Robert P. Sheridan,et al.  Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships , 2015, J. Chem. Inf. Model..

[70]  Bo Fu,et al.  A Survey of Cross-Layer Designs in Wireless Networks , 2014, IEEE Communications Surveys & Tutorials.

[71]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[72]  Janne Riihijärvi,et al.  Cognitive Wireless Networks : Your Network Just Became a Teenager , 2006 .

[73]  Lingfen Sun,et al.  QoE Prediction Model and its Application in Video Quality Adaptation Over UMTS Networks , 2012, IEEE Transactions on Multimedia.

[74]  M. Zorzi,et al.  The challenges of M 2 M massive access in wireless cellular networks , 2015 .

[75]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[76]  Andrzej Osyczka,et al.  7 – Multicriteria optimization for engineering design , 1985 .

[77]  Ryan W. Thomas,et al.  Cognitive networks , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[78]  A. Forster,et al.  Machine Learning Techniques Applied to Wireless Ad-Hoc Networks: Guide and Survey , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[79]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[80]  Andrea Zanella,et al.  SSIM-based video admission control and resource allocation algorithms , 2014, 2014 12th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt).

[81]  Margo I. Seltzer,et al.  File classification in self-* storage systems , 2004 .

[82]  Bessem Sayadi,et al.  Control of Multiple Remote Servers for Quality-Fair Delivery of Multimedia Contents , 2014, IEEE Journal on Selected Areas in Communications.

[83]  Vera Stavroulaki,et al.  5G on the Horizon: Key Challenges for the Radio-Access Network , 2013, IEEE Vehicular Technology Magazine.

[84]  Ian F. Akyildiz,et al.  NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey , 2006, Comput. Networks.

[85]  Henning Sanneck,et al.  LTE Self-Organising Networks (SON): Network Management Automation for Operational Efficiency , 2012 .

[86]  Geoffrey E. Hinton Learning multiple layers of representation , 2007, Trends in Cognitive Sciences.

[87]  BalakrishnanHari,et al.  TCP ex machina , 2013 .

[88]  Yashar Ganjali,et al.  HyperFlow: A Distributed Control Plane for OpenFlow , 2010, INM/WREN.

[89]  Michael I. Jordan Graphical Models , 1998 .

[90]  Kwang-Cheng Chen,et al.  Massive Access Management for QoS Guarantees in 3GPP Machine-to-Machine Communications , 2011, IEEE Communications Letters.

[91]  Nazim Agoulmine,et al.  Autonomic network management principles : from concepts to applications , 2011 .

[92]  Petri Mahonen,et al.  Cognitive Resource Manager , 2007 .

[93]  Allen B. MacKenzie,et al.  Cognitive networks: adaptation and learning to achieve end-to-end performance objectives , 2006, IEEE Communications Magazine.

[94]  Ping Zhang,et al.  Cognitive Wireless Networks , 2015, SpringerBriefs in Electrical and Computer Engineering.

[95]  Rouzbeh Razavi,et al.  A Fuzzy reinforcement learning approach for self-optimization of coverage in LTE networks , 2010, Bell Labs Technical Journal.

[96]  Paul Barford,et al.  A Machine Learning Approach to TCP Throughput Prediction , 2007, IEEE/ACM Transactions on Networking.

[97]  Simon Haykin,et al.  Cognitive radio: brain-empowered wireless communications , 2005, IEEE Journal on Selected Areas in Communications.

[98]  Andrea Zanella,et al.  A machine learning approach to QoE-based video admission control and resource allocation in wireless systems , 2014, 2014 13th Annual Mediterranean Ad Hoc Networking Workshop (MED-HOC-NET).

[99]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[100]  Michael I. Jordan,et al.  Failure diagnosis using decision trees , 2004 .

[101]  Zhong Fan,et al.  Emerging technologies and research challenges for 5G wireless networks , 2014, IEEE Wireless Communications.

[102]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[103]  Ramesh C. Jain,et al.  A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video , 2002, Pattern Recognit..

[104]  Robert W. Heath,et al.  Five disruptive technology directions for 5G , 2013, IEEE Communications Magazine.

[105]  Margo I. Seltzer,et al.  File classification in self-* storage systems , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[106]  Marco Levorato,et al.  A Learning Framework for Cognitive Interference Networks with Partial and Noisy Observations , 2012, IEEE Transactions on Wireless Communications.

[107]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[108]  Andrea Zanella,et al.  Data-driven QoE optimization techniques for multi-user wireless networks , 2015, 2015 International Conference on Computing, Networking and Communications (ICNC).

[109]  Andrea Zanella,et al.  The challenges of M2M massive access in wireless cellular networks , 2015, Digit. Commun. Networks.

[110]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[111]  Andrea Zanella,et al.  A Markov-based framework for handover optimization in HetNets , 2014, 2014 13th Annual Mediterranean Ad Hoc Networking Workshop (MED-HOC-NET).

[112]  Nitish Srivastava,et al.  Multimodal learning with deep Boltzmann machines , 2012, J. Mach. Learn. Res..

[113]  Xiaoli Chu,et al.  Mobility management challenges in 3GPP heterogeneous networks , 2012, IEEE Communications Magazine.

[114]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.