Semantic Textual Similarity (STS) evaluation assesses the degree to which two parts of texts are similar, based on their semantic evaluation. In this paper, we describe three models submitted to STS SemEval 2017. Given two English parts of a text, each of proposed methods outputs the assessment of their semantic similarity. We propose an approach for computing monolingual semantic textual similarity based on an ensemble of three distinct methods. Our model consists of recursive neural network (RNN) text auto-encoders ensemble with supervised a model of vectorized sentences using reduced part of speech (PoS) weighted word embeddings as well as unsupervised a method based on word coverage (TakeLab). Additionally, we enrich our model with additional features that allow disambiguation of ensemble methods based on their efficiency. We have used Multi-Layer Perceptron as an ensemble classifier basing on estimations of trained Gradient Boosting Regressors. Results of our research proves that using such ensemble leads to a higher accuracy due to a fact that each member-algorithm tends to specialize in particular type of sentences. Simple model based on PoS weighted Word2Vec word embeddings seem to improve performance of more complex RNN based auto-encoders in the ensemble. In the monolingual English-English STS subtask our Ensemble based model achieved mean Pearson correlation of .785 compared with human annotators.
[1]
M Abecassis,et al.
Word Frequencies in Written and Spoken English.
,
2010
.
[2]
Jeffrey Dean,et al.
Distributed Representations of Words and Phrases and their Compositionality
,
2013,
NIPS.
[3]
Eneko Agirre,et al.
SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation
,
2016,
*SEMEVAL.
[4]
Jeffrey Pennington,et al.
Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
,
2011,
NIPS.
[5]
Jan Snajder,et al.
TakeLab: Systems for Measuring Semantic Text Similarity
,
2012,
*SEMEVAL.
[6]
Sanja Fidler,et al.
Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books
,
2015,
2015 IEEE International Conference on Computer Vision (ICCV).
[7]
Gaël Varoquaux,et al.
Scikit-learn: Machine Learning in Python
,
2011,
J. Mach. Learn. Res..