Learning chinese word embeddings from character structural information

Abstract Word embedding is a basic task in natural language processing area. Unlike English, Chinese subword units, such as characters, radicals, and components, contain rich semantic information which can be used to enhance word embeddings. However, existing methods neglect the semantic contribution of corresponding subword units to the word. In this work, we employ attention mechanism to capture the semantic structure of Chinese words and propose a novel framework, named Attention-based multi-Layer Word Embedding model(ALWE). We also design an asynchronous strategy for updating embedding and attention efficiently. Our model learns to share subword information between distinct words selectively and adaptively. Experimental results on the word similarity, word analogy, and text classification show that the proposed model outperforms all baselines, especially when words do not appear frequently. Qualitative analysis further demonstrates the superiority of ALWE.