Assessment of text coherence using an ontology‐based relatedness measurement method

This paper proposes a novel method for assessing text coherence. Central to this approach is an ontology‐based representation of text, which captures the level of relatedness between consecutive sentences via ontologies. Our method encompasses annotating text using ontological concepts and assessing text coherence based on relatedness measurement among these concepts. The ontology‐based relatedness measurement method used in this study considers various types of relationships in ontologies and derived relationships via an inference engine for computing relatedness. We hypothesized that rich variety of relationships and inferred facts in ontologies would improve the success of text coherence assessment. Our results demonstrate that the use of ontologies yields to coherence values that have a higher correlation with human ratings.

[1]  Erik Cambria,et al.  Learning short-text semantic similarity with word embeddings and external knowledge sources , 2019, Knowl. Based Syst..

[2]  Ana M. García-Serrano,et al.  A reproducible survey on word embeddings and ontology-based methods for word similarity: Linear combinations outperform the state of the art , 2019, Eng. Appl. Artif. Intell..

[3]  Olga Kononova,et al.  Unsupervised word embeddings capture latent knowledge from materials science literature , 2019, Nature.

[4]  Kyung Sup Kwak,et al.  Transportation sentiment analysis using word embedding and ontology-based topic modeling , 2019, Knowl. Based Syst..

[5]  Korhan Günel,et al.  An empirical study on evolutionary feature selection in intelligent tutors for learning concept detection , 2019, Expert Syst. J. Knowl. Eng..

[6]  Sungyoung Lee,et al.  A knowledge construction methodology to automate case‐based learning using clinical documents , 2019, Expert Syst. J. Knowl. Eng..

[7]  Prashanti Manda,et al.  Comparison of Natural Language Processing Tools for Automatic Gene Ontology Annotation of Scientific Literature , 2018, ICBO.

[8]  Zhongfei Zhang,et al.  Text Coherence Analysis Based on Deep Neural Network , 2017, CIKM.

[9]  Takenobu Tokunaga,et al.  Evaluating text coherence based on semantic similarity graph , 2017, TextGraphs@ACL.

[10]  Honglak Lee,et al.  Sentence Ordering and Coherence Modeling using Recurrent Neural Networks , 2016, AAAI.

[11]  Fatih Yücalar,et al.  Multi‐level reranking approach for bug localization , 2016, Expert Syst. J. Knowl. Eng..

[12]  Fariborz Mahmoudi,et al.  Conceptual feature generation for textual information using a conceptual network constructed from Wikipedia , 2016, Expert Syst. J. Knowl. Eng..

[13]  Mehdi Allahyari,et al.  Automatic Topic Labeling Using Ontology-Based Topic Models , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[14]  M. de Rijke,et al.  Short Text Similarity with Word Embeddings , 2015, CIKM.

[15]  Hao Wang,et al.  Semantic data mining: A survey of ontology-based approaches , 2015, Proceedings of the 2015 IEEE 9th International Conference on Semantic Computing (IEEE ICSC 2015).

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  François Lévy,et al.  Ontology-based Technical Text Annotation , 2014, COLING 2014.

[18]  Erik Cambria,et al.  Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article] , 2014, IEEE Computational Intelligence Magazine.

[19]  P. Broek,et al.  A Cognitive View of Reading Comprehension: Implications for Reading Difficulties , 2014 .

[20]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[21]  Camille Guinaudeau,et al.  Graph-based Local Coherence Modeling , 2013, ACL.

[22]  Kalina Bontcheva,et al.  Microblog-genre noise and impact on semantic annotation accuracy , 2013, HT.

[23]  Murat Osman Ünalir,et al.  A method for ontology-based semantic relatedness measurement , 2013 .

[24]  J. Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[25]  Csongor Nyulas,et al.  BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications , 2011, Nucleic Acids Res..

[26]  J. Hendler,et al.  Semantic Web for the Working Ontologist: Effective Modeling in RDFS and OWL , 2011 .

[27]  S. A. M. Rizvi,et al.  Semantic Annotation Framework For Intelligent Information Retrieval Using KIM Architecture , 2010 .

[28]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[29]  Christiane Fellbaum,et al.  Putting Semantics into WordNet's "Morphosemantic" Links , 2009, LTC.

[30]  Martin Hepp,et al.  GoodRelations: An Ontology for Describing Products and Services Offers on the Web , 2008, EKAW.

[31]  J. Euzenat,et al.  Ontology Matching , 2007, Springer Berlin Heidelberg.

[32]  Yarden Katz,et al.  Pellet: A practical OWL-DL reasoner , 2007, J. Web Semant..

[33]  Kalina Bontcheva,et al.  Semantic Annotation and Human Language Technology , 2006 .

[34]  Sanna-Kaisa Tanskanen.  Collaborating Towards Coherence: Lexical Cohesion in English Discourse , 2006 .

[35]  Mirella Lapata,et al.  Automatic Evaluation of Text Coherence: Models and Representations , 2005, IJCAI.

[36]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[37]  Joseph P. Magliano,et al.  Causal and Semantic Relatedness in Discourse Understanding and Representation , 2005 .

[38]  Timothy W. Finin,et al.  Swoogle: a search and metadata engine for the semantic web , 2004, CIKM '04.

[39]  Kalina Bontcheva,et al.  Evolving GATE to meet new challenges in language engineering , 2004, Natural Language Engineering.

[40]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[41]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[42]  Chaomei Chen,et al.  Mining the Web: Discovering knowledge from hypertext data , 2004, J. Assoc. Inf. Sci. Technol..

[43]  Naoaki Okazaki,et al.  Sentence Extraction by Spreading Activation through Sentence Similarity , 2003 .

[44]  James A. Hendler,et al.  Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential , 2002 .

[45]  Brian McBride,et al.  Jena: A Semantic Web Toolkit , 2002, IEEE Internet Comput..

[46]  Hwee Tou Ng,et al.  A Machine Learning Approach to Coreference Resolution of Noun Phrases , 2001, CL.

[47]  Ann Grafstein,et al.  The linguistic assumptions underlying readability formulae , 2001 .

[48]  Charles A. Perfetti,et al.  Comprehending written language: a blueprint of the reader , 2000 .

[49]  Dekang Lin,et al.  An Information-Theoretic Definition of Similarity , 1998, ICML.

[50]  W. Kintsch.  Comprehension: A Paradigm for Cognition , 1998 .

[51]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[52]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[53]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[54]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[55]  R. Ratcliff,et al.  Spreading activation versus compound cue accounts of priming: mediated priming revisited. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[56]  R. Ratcliff,et al.  Inference during reading. , 1992, Psychological review.

[57]  B. K. Britton,et al.  Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. , 1991 .

[58]  M. Just,et al.  The psychology of reading and language comprehension , 1986 .

[59]  A. Tversky.  Features of Similarity , 1977 .

[60]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[61]  Gouda I. Salama,et al.  A Novel Approach for Ontology-Based Feature Vector Generation for Web Text Document Classification , 2018, Int. J. Softw. Innov..

[62]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1980 A theory of reading : From eye fixations to comprehension , 2017 .

[63]  Shafiq R. Joty,et al.  A Neural Local Coherence Model , 2017, ACL.

[64]  Lawrence Hunter,et al.  Gold-Standard Ontology-Based Annotation of Concepts in Biomedical Text in the CRAFT Corpus: Updates and Extensions , 2016, ICBO/BioCreative.

[65]  Maria Teresa Pazienza,et al.  A Flexible Approach to Semantic Annotation Systems for Web Content , 2015, Intell. Syst. Account. Finance Manag..

[66]  Alberto Trombetta,et al.  BPMN: An introduction to the standard , 2012, Comput. Stand. Interfaces.

[67]  Joseph P. Magliano,et al.  Chapter 9 Toward a Comprehensive Model of Comprehension , 2009 .

[68]  M. Sabou,et al.  WATSON: a gateway for the semantic web , 2007 .

[69]  Amit P. Sheth,et al.  OntoQA: Metric-Based Ontology Quality Analysis , 2005 .

[70]  Otto H. MacLin,et al.  Cognitive psychology, 7th ed. , 2005 .

[71]  Jordan L. Boyd-Graber,et al.  Adding dense, weighted connections to WordNet , 2005 .

[72]  A. Bernstein,et al.  SimPack: A Generic Java Library for Similarity Measures in Ontologies , 2005 .

[73]  Graeme Hirst,et al.  Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures , 2004 .

[74]  Leo Obrst,et al.  The Semantic Web: A Guide to the Future of XML, Web Services and Knowledge Management , 2003 .

[75]  Deborah L. McGuinness,et al.  Ontologies Come of Age , 2003, Spinning the Semantic Web.

[76]  H. Cunningham,et al.  A framework and graphical development environment for robust NLP tools and applications. , 2002, ACL 2002.

[77]  H. Cunningham,et al.  Developing Language Processing Components with GATE , 2001 .

[78]  J. Hendler,et al.  The Semantic Web: A new form of Web content that is meaningful to computers will unleash a revolutio , 2001 .

[79]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[80]  Peter W. Foltz,et al.  The Measurement of Textual Coherence with Latent Semantic Analysis. , 1998 .

[81]  Martin Chodorow,et al.  Combining local context and wordnet similarity for word sense identification , 1998 .

[82]  W. Kintsch,et al.  Reading comprehension and readability in educational practice and psychological theory , 1979 .

[83]  Dragomir R. Radev,et al.  of the Association for Computational Linguistics , 2022 .