The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning

In this work, we focus on an analogical reasoning task that contains rich compositional structures, Raven's Progressive Matrices (RPM). To discover compositional structures of the data, we propose the Scattering Compositional Learner (SCL), an architecture that composes neural networks in a sequence. Our SCL achieves state-of-the-art performance on two RPM datasets, with a 48.7% relative improvement on Balanced-RAVEN and 26.4% on PGM over the previous state-of-the-art. We additionally show that our model discovers compositional representations of objects' attributes (e.g., shape color, size), and their relationships (e.g., progression, union). We also find that the compositional representation makes the SCL significantly more robust to test-time domain shifts and greatly improves zero-shot generalization to previously unseen analogies.

[1]  Christopher D. Manning,et al.  Compositional Attention Networks for Machine Reasoning , 2018, ICLR.

[2]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Feng Gao,et al.  RAVEN: A Dataset for Relational and Analogical Visual REasoNing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Felix Hill,et al.  Learning to Make Analogies by Contrasting Abstract Relational Structure , 2019, ICLR.

[5]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[6]  Stéphane Mallat,et al.  Classification with invariant scattering representations , 2011, 2011 IEEE 10th IVMSP Workshop: Perception and Visual Signal Analysis.

[7]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[8]  Yixin Zhu,et al.  Learning Perceptual Inference by Contrasting , 2019, NeurIPS.

[9]  Ernest Lepore,et al.  The compositionality papers , 2002 .

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Thomas L. Griffiths,et al.  Automatically Composing Representation Transformations as a Means for Generalization , 2018, ICLR.

[13]  Melanie Mitchell,et al.  The Copycat project: a model of mental fluidity and analogy-making , 1995 .

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Kecheng Zheng,et al.  Abstract Reasoning with Distracting Features , 2019, NeurIPS.

[16]  Aaron C. Courville,et al.  Systematic Generalization: What Is Required and Can It Be Learned? , 2018, ICLR.

[17]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ke Wang,et al.  Automatic Generation of Raven's Progressive Matrices , 2015, IJCAI.

[19]  J. Raven,et al.  Raven Progressive Matrices , 2003 .

[20]  Marco Baroni,et al.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks , 2017, ICML.

[21]  Susanne M. Jaeggi,et al.  Improving fluid intelligence with training on working memory: a meta-analysis , 2008, Psychonomic Bulletin & Review.

[22]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[23]  Xiao Wang,et al.  Measuring Compositional Generalization: A Comprehensive Method on Realistic Data , 2019, ICLR.

[24]  Marco Baroni,et al.  Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks , 2018, BlackboxNLP@EMNLP.

[25]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[26]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Yoshua Bengio,et al.  CLOSURE: Assessing Systematic Generalization of CLEVR Models , 2019, ViGIL@NeurIPS.

[29]  Felix Hill,et al.  Measuring abstract reasoning in neural networks , 2018, ICML.

[30]  Xianglong Liu,et al.  Hierarchical Rule Induction Network for Abstract Visual Reasoning , 2020, ArXiv.

[31]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[32]  Marc Brockschmidt,et al.  Differentiable Programs with Neural Libraries , 2016, ICML.

[33]  Pietro Liò,et al.  Abstract Diagrammatic Reasoning with Multiplex Graph Networks , 2020, ICLR.

[34]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[35]  G. Marcus.  Rethinking Eliminative Connectionism , 1998, Cognitive Psychology.

[36]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[37]  M A Just,et al.  From the SelectedWorks of Marcel Adam Just 1990 What one intelligence test measures : A theoretical account of the processing in the Raven Progressive Matrices Test , 2016 .