A Statistical Approach for the Best Deep Neural Network Configuration for Arabic Language Processing

The widespread of the computer technology and the Internet lead to a massive amount of textual information being available in written Arabic. This that more is available, it becomes more difficult to extract the relevant information. To meet this challenge, many researchers are directed to the development of information retrieval systems based on syntactic and semantic parsing. In Arabic, this field is restricted by the lack of labeled datasets. Thus, it is important to build systems for part-of-speech tagging and language modeling and use their results for further syntactic and semantic parsing in fields like chunking, semantic role labeling, information extraction, named entity recognition and statistical machine translation. Deep neural networks have proved efficient in fields like imaging or acoustics and recently in natural language processing. In this study, we used the Taguchi method to find the optimal parameter combination for a deep neural network architecture. Therefore, the neural network obtained the most accurate results. The main use of the Taguchi method in our work is to help us to choose the best context which is the number of words before and after the word on which the training is made.

[1]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[2]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[3]  Kevin Duh,et al.  POS Tagging of Dialectal Arabic: A Minimally Supervised Approach , 2005, SEMITIC@ACL.

[4]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[5]  Daniel Jurafsky,et al.  Automatic Tagging of Arabic Text: From Raw Text to Base Phrase Chunks , 2004, NAACL.

[6]  Ilana Heintz Arabic Language Modeling with Finite State Transducers , 2008, ACL.

[7]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[8]  Nigel Collier,et al.  Use of Support Vector Machines in Extended Named Entity Recognition , 2002, CoNLL.

[9]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[10]  Andreas Stolcke,et al.  Morphology-based language modeling for arabic speech recognition , 2004, INTERSPEECH.

[11]  Virginia Teller Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. , 2000 .

[12]  Christian Igel,et al.  An Introduction to Restricted Boltzmann Machines , 2012, CIARP.

[13]  S. Khoja,et al.  APT: Arabic Part-of-speech Tagger , 2001 .

[14]  Mahmoud El-Haj,et al.  KALIMAT a multipurpose Arabic corpus , 2013 .