ScatterNet hybrid frameworks for deep learning

Image understanding is the task of interpreting images by effectively solving the individual tasks of object recognition and semantic image segmentation. An image understanding system must have the capacity to distinguish between similar looking image regions while being invariant in its response to regions that have been altered by the appearance-altering transformation. The fundamental challenge for any such system lies within this simultaneous requirement for both invariance and specificity. Many image understanding systems have been proposed that capture geometric properties such as shapes, textures, motion and 3D perspective projections using filtering, non-linear modulus, and pooling operations. Deep learning networks ignore these geometric considerations and compute descriptors having suitable invariance and stability to geometric transformations using (end-to-end) learned multi-layered network filters. These deep learning networks in recent years have come to dominate the previously separate fields of research in machine learning, computer vision, natural language understanding and speech recognition. Despite the success of these deep networks, there remains a fundamental lack of understanding in the design and optimization of these networks which makes it difficult to develop them. Also, training of these networks requires large labeled datasets which in numerous applications may not be available. In this dissertation, we propose the ScatterNet Hybrid Framework for Deep Learning that is inspired by the circuitry of the visual cortex. The framework uses a handcrafted front-end, an unsupervised learning based middle-section, and a supervised back-end to rapidly learn hierarchical features from unlabelled data. Each layer in the proposed framework is automatically optimized to produce the desired computationally efficient architecture. The term ‘Hybrid’ is coined because the framework uses both unsupervised as well as supervised learning. We propose two hand-crafted front-ends that can extract locally invariant features from the input signals. Next, two ScatterNet Hybrid Deep Learning (SHDL) networks (a generative and a deterministic) were introduced by combining the proposed frontends with two unsupervised learning modules which learn hierarchical features. These

[1]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[2]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[3]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Rob Fergus,et al.  Stochastic Pooling for Regularization of Deep Convolutional Neural Networks , 2013, ICLR.

[5]  Stéphane Mallat,et al.  Rotation, Scaling and Deformation Invariant Scattering for Texture Discrimination , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[7]  Stéphane Mallat,et al.  Deep roto-translation scattering for object classification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Zhanyi Hu,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTION ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1 Rotationally Invariant Descript , 2011 .

[9]  Antonio Jose Rodríguez-Sánchez,et al.  Hierarchical Object Representations in the Visual Cortex and Computer Vision , 2016 .

[10]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[11]  Shuicheng Yan,et al.  Multi-Path Feedback Recurrent Neural Networks for Scene Parsing , 2016, AAAI.

[12]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[13]  Nick G. Kingsbury,et al.  Multi-Resolution Dual-Tree Wavelet Scattering Network for Signal Classification , 2016, ArXiv.

[14]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[15]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[16]  Yong Xu,et al.  A new texture descriptor using multifractal analysis in multi-orientation wavelet pyramid , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Joakim Andén,et al.  Joint time-frequency scattering for audio classification , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[18]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[19]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[20]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[21]  Ronan Collobert,et al.  Recurrent Convolutional Neural Networks for Scene Parsing , 2013, ArXiv.

[22]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[23]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[24]  Sofie Pollin,et al.  When Autonomous Drones Meet Driverless Cars , 2018, MobiSys.

[25]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[27]  Antonello Pasini,et al.  Artificial neural networks for small dataset analysis. , 2015, Journal of thoracic disease.

[28]  Xin Li,et al.  Adaptive Active Learning for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Razvan Pascanu,et al.  Memory-based Parameter Adaptation , 2018, ICLR.

[30]  A. Singh,et al.  Constructive Learning for Human-Robot Interaction , 2013, IEEE Potentials.

[31]  Yanjun Qi,et al.  Unsupervised Feature Learning by Deep Sparse Coding , 2013, SDM.

[32]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[33]  Stephen Gould,et al.  Decomposing a scene into geometric and semantically consistent regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[34]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[35]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[36]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[37]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[39]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[40]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[41]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Devendra Patil,et al.  Eye in the Sky: Real-Time Drone Surveillance System (DSS) for Violent Individuals Identification Using ScatterNet Hybrid Deep Learning Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[43]  Guosheng Lin,et al.  Discriminative Training of Deep Fully Connected Continuous CRFs With Task-Specific Loss , 2016, IEEE Transactions on Image Processing.

[44]  Sukriti Jain,et al.  A Novel Method to Improve Model fitting for Stock Market Prediction , 2013 .

[45]  Gilles Wainrib,et al.  Mathematical modeling of lymphocytes selection in the germinal center , 2017, Journal of mathematical biology.

[46]  Justin Domke,et al.  Learning Graphical Model Parameters with Approximate Marginal Inference , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[48]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Kristin J. Dana,et al.  Compact representation of bidirectional texture functions , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[52]  Shafiee Mohammad Javad,et al.  The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations Through Sexual Evolutionary Synthesis , 2017 .

[53]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[54]  Wenbin Zou,et al.  Semantic segmentation via sparse coding over hierarchical regions , 2012, 2012 19th IEEE International Conference on Image Processing.

[55]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[56]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[57]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[58]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[59]  N. Kingsbury.  Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[60]  Arati Dandavate,et al.  Semantic Texton Forests for Image Categorization and Segmentation , 2018, IJARCCE.

[61]  P. Dayan,et al.  An unsupervised learning model of neural plasticity: Orientation selectivity in goggle-reared kittens , 2007, Vision Research.

[62]  Miguel Á. Carreira-Perpiñán,et al.  Multiscale conditional random fields for image labeling , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[63]  David G. Lowe,et al.  Spatially Local Coding for Object Recognition , 2012, ACCV.

[64]  Thomas Villmann,et al.  Kernelized vector quantization in gradient-descent learning , 2015, Neurocomputing.

[65]  Haibin Ling,et al.  Multi-Level Contextual RNNs With Attention Model for Scene Labeling , 2016, IEEE Transactions on Intelligent Transportation Systems.

[66]  Jing Liu,et al.  Weakly Supervised RBM for Semantic Segmentation , 2015, IJCAI.

[67]  Nick G. Kingsbury,et al.  Efficient Convolutional Network Learning Using Parametric Log Based Dual-Tree Wavelet ScatterNet , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[68]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  D. C. Essen,et al.  Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey. , 1996, Journal of neurophysiology.

[70]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[71]  David D. Cox,et al.  Opinion TRENDS in Cognitive Sciences Vol.11 No.8 Untangling invariant object recognition , 2022 .

[72]  Weihong Deng,et al.  Very deep convolutional neural network based image classification using small training sample size , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[73]  Michelle Karg,et al.  Learning Efficient Deep Feature Representations via Transgenerational Genetic Transmission of Environmental Information During Evolutionary Synthesis of Deep Neural Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[74]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[75]  T. Poggio,et al.  A feedforward theory of visual cortex accounts for human performance in rapid categorization , 2006 .

[76]  Yoshua Bengio,et al.  Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery , 2012, ArXiv.

[77]  Andrew Y. Ng,et al.  The Importance of Encoding Versus Training with Sparse Coding and Vector Quantization , 2011, ICML.

[78]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[79]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.

[80]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[81]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[82]  Nick G. Kingsbury,et al.  Determining Multiscale Image Feature Angles from Complex Wavelet Phases , 2005, ICIAR.

[83]  Patrick Pérez,et al.  The Semantic Paintbrush: Interactive 3D Mapping and Recognition in Large Outdoor Spaces , 2015, CHI.

[84]  Stephan Tschechne,et al.  Hierarchical representation of shapes in visual cortex—from localized features to figural shape segregation , 2014, Front. Comput. Neurosci..

[85]  Amarjot Singh,et al.  Texture and Structure Incorporated ScatterNet Hybrid Deep Learning Network (TS-SHDL) for Brain Matter Segmentation , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[86]  Colin Raffel,et al.  Realistic Evaluation of Deep Semi-Supervised Learning Algorithms , 2018, NeurIPS.

[87]  Yani Ioannou,et al.  Structural priors in deep neural networks , 2018 .

[88]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[89]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[90]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[91]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[92]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[93]  Jean Ponce,et al.  Learning mid-level features for recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[94]  Alfred O. Hero,et al.  Efficient learning of sparse, distributed, convolutional feature representations for object recognition , 2011, 2011 International Conference on Computer Vision.

[95]  Antonio J. Plaza,et al.  On the use of small training sets for neural network-based characterization of mixed pixels in remotely sensed hyperspectral images , 2009, Pattern Recognit..

[96]  Boleslaw K. Szymanski,et al.  Taming the Curse of Dimensionality in Kernels and Novelty Detection , 2004, WSC.

[97]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[98]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[99]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[100]  Nick Kingsbury,et al.  The dual-tree complex wavelet transform: a new technique for shift invariance and directional filters , 1998 .

[101]  Andrew Y. Ng,et al.  Learning Feature Representations with K-Means , 2012, Neural Networks: Tricks of the Trade.

[102]  Alexei A. Efros,et al.  Unsupervised discovery of visual object class hierarchies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[103]  Ce Liu,et al.  Unsupervised Joint Object Discovery and Segmentation in Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[104]  Kurt Hornik,et al.  Support Vector Machines in R , 2006 .

[105]  William Eberle,et al.  Instance selection by genetic-based biological algorithm , 2015, Soft Comput..

[106]  Thomas Brox,et al.  Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[107]  Concetto Spampinato,et al.  Semi Supervised Semantic Segmentation Using Generative Adversarial Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[108]  Nick G. Kingsbury,et al.  Dual-Tree wavelet scattering network with parametric log transformation for object classification , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[109]  Mohak Shah,et al.  Effective Building Block Design for Deep Convolutional Neural Networks using Search , 2018, ArXiv.

[110]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[111]  Thomas Serre,et al.  Realistic Modeling of Simple and Complex Cell Tuning in the HMAX Model, and Implications for Invariant Object Recognition in Cortex , 2004 .

[112]  Hui Ma,et al.  Partial discharge pattern recognition using multiscale feature extraction and support vector machine , 2013, 2013 IEEE Power & Energy Society General Meeting.

[113]  Joan Bruna.  Scattering Representations for Recognition , 2013 .

[114]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[115]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[116]  Xi Zhang,et al.  Learning from Synthetic Data Using a Stacked Multichannel Autoencoder , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[117]  Y-Lan Boureau,et al.  Learning Convolutional Feature Hierarchies for Visual Recognition , 2010, NIPS.

[118]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[119]  Andreas Geiger,et al.  Automatic camera and range sensor calibration using a single shot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[120]  Trevor Darrell,et al.  Beyond spatial pyramids: Receptive field learning for pooled image features , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[121]  H. T. Kung,et al.  Stable and Efficient Representation Learning with Nonnegativity Constraints , 2014, ICML.

[122]  Daphne Koller,et al.  Active learning: theory and applications , 2001 .

[123]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[124]  Nikolaos Papanikolopoulos,et al.  Multi-class active learning for image classification , 2009, CVPR.

[125]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[126]  Wolfram Burgard,et al.  Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion , 2016, ISER.

[127]  R. Desimone,et al.  Stimulus-selective properties of inferior temporal neurons in the macaque , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[128]  Jun Yu,et al.  Unsupervised image segmentation via Stacked Denoising Auto-encoder and hierarchical patch indexing , 2018, Signal Process..

[129]  Alexander G. Huth,et al.  Attention During Natural Vision Warps Semantic Representation Across the Human Brain , 2013, Nature Neuroscience.

[130]  Jitendra Malik,et al.  Learning to Optimize , 2016, ICLR.

[131]  B. Wandell.  Foundations of vision , 1995 .

[132]  Honglak Lee,et al.  Learning Invariant Representations with Local Transformations , 2012, ICML.

[133]  Nick G. Kingsbury,et al.  Scatternet hybrid deep learning (SHDL) network for object classification , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[134]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[135]  Leif E. Peterson.  K-nearest neighbor , 2009, Scholarpedia.

[136]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[137]  Dieter Fox,et al.  Kernel Descriptors for Visual Recognition , 2010, NIPS.

[138]  Armand Joulin,et al.  Unsupervised Learning by Predicting Noise , 2017, ICML.

[139]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[140]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[141]  Amarjot Singh,et al.  Aerial Scene Understanding Using Deep Wavelet Scattering Network and Conditional Random Field , 2016, ECCV Workshops.

[142]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[143]  Jacek Tabor,et al.  Extreme entropy machines: robust information theoretic classification , 2015, Pattern Analysis and Applications.

[144]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[145]  Alexander Zien,et al.  Label Propagation and Quadratic Criterion , 2006 .

[146]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[147]  Haichao Zhu,et al.  A New Method to Assist Small Data Set Neural Network Learning , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[148]  Stéphane Mallat,et al.  Combined scattering for rotation invariant texture analysis , 2012, ESANN.

[149]  Andrew Zisserman,et al.  Efficient Visual Search of Videos Cast as Text Retrieval , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[150]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[151]  Hassiba Nemmour,et al.  Fuzzy Integral Combination of One-Class Classifiers Designed for Multi-class Classification , 2014, ICIAR.

[152]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[153]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[154]  Xiao Liu,et al.  Semi-supervised Node Splitting for Random Forest Construction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[155]  T. Blumensath,et al.  On the Difference Between Orthogonal Matching Pursuit and Orthogonal Least Squares , 2007 .

[156]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[157]  Vacius Jusas,et al.  Convolutional Neural Networks for Four-Class Motor Imagery Data Classification , 2017, IDC.

[158]  Pietro Perona,et al.  Entropy-based active learning for object recognition , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[159]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[160]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[161]  Nick G. Kingsbury,et al.  Visualizing and improving scattering networks , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[162]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[163]  Marleen de Bruijne,et al.  Supervised Image Segmentation across Scanner Protocols: A Transfer Learning Approach , 2012, MLMI.

[164]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[165]  Luc Van Gool,et al.  Unsupervised High-level Feature Learning by Ensemble Projection for Semi-supervised Image Classification and Image Clustering , 2016, ArXiv.

[166]  G. F. Cooper,et al.  Development of the Brain depends on the Visual Environment , 1970, Nature.

[167]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[168]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[169]  S. Mallat,et al.  Invariant Scattering Convolution Networks , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[170]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[171]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[172]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.