GPU-based Self-Organizing Maps for Post-Labeled Few-Shot Unsupervised Learning

Few-shot classification is a challenge in machine learning where the goal is to train a classifier using a very limited number of labeled examples. This scenario is likely to occur frequently in real life, for example when data acquisition or labeling is expensive. In this work, we consider the problem of post-labeled few-shot unsupervised learning, a classification task where representations are learned in an unsupervised fashion, to be later labeled using very few annotated examples. We argue that this problem is very likely to occur on the edge, when the embedded device directly acquires the data, and the expert needed to perform labeling cannot be prompted often. To address this problem, we consider an algorithm consisting of the concatenation of transfer learning with clustering using Self-Organizing Maps (SOMs). We introduce a TensorFlow-based implementation to speed-up the process in multi-core CPUs and GPUs. Finally, we demonstrate the effectiveness of the method using standard off-the-shelf few-shot classification benchmarks.

[1]  Emilio Del-Moral-Hernandez,et al.  An FPGA distributed implementation model for embedded SOM with on-line learning , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[2]  James T. Kwok,et al.  Generalizing from a Few Examples , 2019, ACM Comput. Surv..

[3]  Benoît Miramond,et al.  Energy consumption minimization on LoRaWAN sensor network by using an Artificial Neural Network based application , 2019, 2019 IEEE Sensors Applications Symposium (SAS).

[4]  Andres Upegui,et al.  Self-organizing neurons: toward brain-inspired unsupervised learning , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[5]  Bernard Girau,et al.  Neuromorphic hardware as a self-organizing computing system , 2018, ArXiv.

[6]  Vincent Gripon,et al.  Exploiting Unsupervised Inputs for Accurate Few-Shot Classification , 2020, ArXiv.

[7]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[8]  Richard Hurley,et al.  Scalability of Self-organizing Maps on a GPU cluster using OpenCL and CUDA , 2012 .

[9]  Nikos Komodakis,et al.  Generating Classification Weights With GNN Denoising Autoencoders for Few-Shot Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[11]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[12]  Damminda Alahakoon,et al.  Machine learning to support social media empowered patients in cancer care and cancer treatment decisions , 2018, PloS one.

[13]  Irem Dikmen,et al.  Comparing the performance of traditional cluster analysis, self-organizing maps and fuzzy C-means method for strategic grouping , 2009, Expert Syst. Appl..

[14]  Jinlu Liu,et al.  Prototype Rectification for Few-Shot Learning , 2020, ECCV.

[15]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[16]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[17]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[18]  F. C. Moraes,et al.  Parallel High Dimensional Self Organizing Maps Using CUDA , 2012, 2012 Brazilian Robotics Symposium and Latin American Robotics Symposium.

[19]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[20]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[21]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[22]  Laurent Rodriguez,et al.  A distributed cellular approach of large scale SOM models for hardware implementation , 2018, 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS).

[23]  Erkki Oja,et al.  Engineering applications of the self-organizing map , 1996, Proc. IEEE.

[24]  Gui-Bin Bian,et al.  Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications , 2018, IEEE Access.

[25]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[26]  Xinghuo Yu,et al.  Intelligent Detection of Driver Behavior Changes for Effective Coordination Between Autonomous and Human Driven Vehicles , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[27]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[28]  Vadlamani Ravi,et al.  CUDA-Self-Organizing feature map based visual sentiment analysis of bank customer complaints for Analytical CRM , 2019, ArXiv.