Collective dynamics in complex networks for machine learning

VERRI, F. A. N. Collective dynamics in complex networks for machine learning. 2018. 95 p. Tese (Doutorado em Ciências – Ciências de Computação e Matemática Computacional) – Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Carlos – SP, 2018. Machine learning enables machines to learn automatically from data. In literature, graph-based methods have received increasing attention due to their ability to learn from both local and global information. In these methods, each data instance is represented by a vertex and is linked to other vertices according to a predefined affinity rule. However, they usually have unfeasible time cost for large problems. To overcome this problem, techniques can employ a heuristic to find suboptimal solutions in a feasible time. Early heuristic optimization methods exploit nature-inspired collective processes, such as ants looking for food sources and swarms of bees. Nowadays, advances in the field of complex systems provide powerful tools to assess and to understand dynamical systems. Complex networks, which are graphs with nontrivial topology, are among these theoretical tools capable of describing the interplay of topology, structure, and dynamics of complex systems. Therefore, machine learning methods based on complex networks and collective dynamics have been proposed. They encompass three steps. First, a complex network is constructed from the input data. Then, the simulation of a distributed collective system in the network generates rich information. Finally, the collected information is used to solve the learning problem. The coordination of the individuals in the system permit to achieve dynamics that is far more complex than the behavior of single individuals. In this research, I have explored collective dynamics in machine learning tasks, both in unsupervised and semi-supervised scenarios. Specifically, I have proposed a new collective system of competing particles that shifts the traditional vertex-centric dynamics to a more informative edge-centric one. Moreover, it is the first particle competition system applied in machine learning task that has deterministic behavior. Results show several advantages of the edge-centric model, including the ability to acquire more information about overlapping areas, a better exploration behavior, and a faster convergence time. Also, I have proposed a new network formation technique that is not based on similarity and has low computational cost. Since addition and removal of samples in the network is cheap, it can be used in real-time application. Finally, I have conducted analytical investigations of a flocking-like system that was needed to guarantee the expected behavior in community detection tasks. In conclusion, the result of the research contributes to many areas of machine learning and complex systems.

[1]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[2]  Xiaojin Zhu,et al.  Some new directions in graph-based semi-supervised learning , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[3]  Chang Honghao,et al.  Community detection using Ant Colony Optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[4]  Keping Long,et al.  On Swarm Intelligence Inspired Self-Organized Networking: Its Bionic Mechanisms, Designing Principles and Optimization Approaches , 2014, IEEE Communications Surveys & Tutorials.

[5]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[6]  T. Vicsek,et al.  Collective Motion , 1999, physics/9902023.

[7]  D. Lusseau,et al.  The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations , 2003, Behavioral Ecology and Sociobiology.

[8]  L. Hubert,et al.  Comparing partitions , 1985 .

[9]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Shuang Yu,et al.  PE-PUC: A Graph Based PU-Learning Approach for Text Classification , 2007, MLDM.

[11]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[12]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[13]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[14]  Liang Zhao,et al.  Semi-supervised learning guided by the modularity measure in complex networks , 2012, Neurocomputing.

[15]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[16]  Liang Zhao,et al.  Network Unfolding Map by Vertex-Edge Dynamics Modeling , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  E Oh,et al.  Modular synchronization in complex networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Liang Zhao,et al.  Particle competition for complex network community detection. , 2008, Chaos.

[19]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[20]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[21]  Jurgen Kurths,et al.  Synchronization in complex networks , 2008, 0805.2976.

[22]  I. Aranson,et al.  Collective motion of self-propelled particles with memory. , 2015, Physical review letters.

[23]  Jordi Vitrià,et al.  Weighted Bagging for Graph Based One-Class Classifiers , 2010, MCS.

[24]  Liang Zhao,et al.  Features of edge-centric collective dynamics in machine learning tasks , 2016 .

[25]  Rolf T. Wigand,et al.  Community Detection in Complex Networks: Multi-objective Enhanced Firefly Algorithm , 2013, Knowl. Based Syst..

[26]  Fabricio A. Breve,et al.  Interactive image segmentation using particle competition and cooperation , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[27]  James T. Kwok,et al.  Scaling Up Graph-Based Semisupervised Learning via Prototype Vector Machines , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  A flocking-like technique to perform semi-supervised learning , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[29]  Witold Pedrycz,et al.  Entropic One-Class Classifiers , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[30]  F. Radicchi,et al.  Benchmark graphs for testing community detection algorithms. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.