Humans Keep It One Hundred: an Overview of AI Journey

Artificial General Intelligence (AGI) is showing growing performance in numerous applications - beating human performance in Chess and Go, using knowledge bases and text sources to answer questions (SQuAD) and even pass human examination (Aristo project). In this paper, we describe the results of AI Journey, a competition of AI-systems aimed to improve AI performance on knowledge bases, reasoning and text generation. Competing systems pass the final native language exam (in Russian), including versatile grammar tasks (test and open questions) and an essay, achieving a high score of 69%, with 68% being an average human result. During the competition, a baseline for the task and essay parts was proposed, and 80+ systems were submitted, showing different approaches to task understanding and reasoning. All the data and solutions can be found on github https://github.com/sberbank-ai/combined_solution_aij2019

[1]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[2]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[3]  Mikhail Korobov,et al.  Morphological Analyzer and Generator for Russian and Ukrainian Languages , 2015, AIST.

[4]  Ai Kawazoe,et al.  Overview of Todai Robot Project and Evaluation Framework of its NLP-based Problem Solving , 2014, LREC.

[5]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Pushmeet Kohli,et al.  Analysing Mathematical Reasoning Abilities of Neural Models , 2019, ICLR.

[8]  Ben Goertzel,et al.  Artificial General Intelligence: Concept, State of the Art, and Future Prospects , 2009, J. Artif. Gen. Intell..

[9]  Miguel Cazorla,et al.  ImageCLEF 2014: Overview and Analysis of the Results , 2014, CLEF.

[10]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[11]  Oren Etzioni,et al.  Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , 2018, ArXiv.

[12]  Jake VanderPlas,et al.  A Practical Taxonomy of Reproducibility for Machine Learning Research , 2018 .

[13]  Ali Farhadi,et al.  IQA: Visual Question Answering in Interactive Environments , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[15]  Peter Clark.  Elementary School Science and Math Tests as a Driver for AI: Take the Aristo Challenge! , 2015, AAAI.

[16]  Oren Etzioni,et al.  From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project , 2019, AI Mag..

[17]  Jun Zhao,et al.  Which is the Effective Way for Gaokao: Information Retrieval or Neural Networks? , 2017, EACL.

[18]  Tatiana Likhomanenko,et al.  Improving reproducibility of data science experiments , 2015 .

[19]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[20]  A M Turing,et al.  Computing Machinery and Intelligence A.M. Turing , 2007 .

[21]  Milan Straka,et al.  Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe , 2017, CoNLL.

[22]  Matthias Hagen,et al.  Overview of the 1st international competition on plagiarism detection , 2009 .

[23]  Nan Hua,et al.  Universal Sentence Encoder , 2018, ArXiv.

[24]  Mikhail Arkhipov,et al.  Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language , 2019, ArXiv.

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  Eliza Strickland,et al.  Can an AI get into the University of Tokyo , 2013 .

[27]  Yuzhong Qu,et al.  Taking Up the Gaokao Challenge: An Information Retrieval Approach , 2016, IJCAI.