What’s in a Gist? Towards an Unsupervised Gist Representation for Few-Shot Large Document Classification

The gist can be viewed as an abstract concept that represents only the quintessential meaning derived from a single or multiple sources of information. We live in an age where vast quantities of information are widely available and easily accessible. Identifying the gist contextualises information which facilitates the fast disambiguation and prediction of related concepts bringing about a set of natural relationships defined between information sources. In this paper, we investigate and introduce a novel unsupervised gist extraction and quantification framework that represents a computational form of the gist based on notions from fuzzy trace theory. To evaluate our purposed framework, we apply the gist to the task of semantic similarity, specifically to few-shot large document classification where documents on average have a large number of words. The results show our proposed gist representation can effectively capture the essential information from a text document while dramatically reducing the features used.

[1]  Karen Sparck Jones.  A statistical interpretation of term specificity and its application in retrieval , 1972 .

[2]  V. Reyna,et al.  Gist is the grist: Fuzzy-trace theory and the new intuitionism☆ , 1990 .

[3]  Donia Scott,et al.  A Discourse Model for Gist Preservation , 1996, SBIA.

[4]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[5]  V. Reyna,et al.  Fuzzy-trace theory: dual processes in memory, reasoning, and cognitive neuroscience. , 2001, Advances in child development and behavior.

[6]  J. Neuschatz,et al.  The effect of memory schemata on memory and the phenomenological experience of naturalistic situations , 2002 .

[7]  Maria das Graças Volpe Nunes,et al.  GistSumm: A Summarization Tool Based on a New Extractive Method , 2003, PROPOR.

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[10]  Juliana K. Leding,et al.  Global gist extraction in children and adults , 2006, Memory.

[11]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[13]  Cordelia Schmid,et al.  Evaluation of GIST descriptors for web-scale image search , 2009, CIVR '09.

[14]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[15]  Marlène Abadie,et al.  Gist Memory in the Unconscious-Thought Effect , 2013, Psychological science.

[16]  G. Zipf.  Selected Studies of the Principle of Relative Frequency in Language , 2014 .

[17]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[18]  Matt J. Kusner,et al.  From Word Embeddings To Document Distances , 2015, ICML.

[19]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[20]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[21]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[22]  Vikas Raunak.  Simple and Effective Dimensionality Reduction for Word Embeddings , 2017 .

[23]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[24]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[25]  Shuyue Hu,et al.  To be Big Picture Thinker or Detail-Oriented?: Utilizing Perceived Gist Information to Achieve Efficient Convention Emergence with Bilateralism and Multilateralism , 2019, AAMAS.

[26]  M. Lee.  A Comparison of Machine Measures of Text Document Similarity with Human Judgments , 2022 .