Revisiting the Roles of “Text” in Text Games

Text games present opportunities for natural 001 language understanding (NLU) methods to 002 tackle reinforcement learning (RL) challenges. 003 However, recent work has questioned the ne- 004 cessity of NLU by showing random text hashes 005 could perform decently. In this paper, we pur- 006 sue a fine-grained investigation into the roles of 007 text in the face of different RL challenges, and 008 reconcile that semantic and non-semantic lan- 009 guage representations could be complementary 010 rather than contrasting. Concretely, we propose 011 a simple scheme to extract relevant contextual 012 information into an approximate state hash as 013 extra input for an RNN-based text agent. Such 014 a lightweight plug-in achieves competitive per- 015 formance with state-of-the-art text agents using 016 advanced NLU techniques such as knowledge 017 graph and passage retrieval, suggesting non- 018 NLU methods might suffice to tackle the chal- 019 lenge of partial observability . However, if we 020 remove RNN encoders and use approximate or 021 even ground-truth state hash alone, the model 022 performs miserably, which confirms the impor- 023 tance of semantic function approximation to 024 tackle the challenge of combinatorially large 025 observation and action spaces . Our findings 026 and analysis provide new insights for designing 027 better text game task setups and agents. 028