Stochastic Optimization for CRF Autoencoders

The goal of this project is to implement existing stochastic optimization methods in the context of conditional random field (CRF) autoencoders [1]. CRF autoencoders are a class of probabilistic models which was designed to address unsupervised and semi-supervised problems in natural language processing. However, for concreteness, we will focus on a particular instantiation of CRF autoencoders for the classic problem of bitext word alignment [4]. In this milestone, we experimented with several variations of stochastic gradient descent (see §3.1 for details). By the end of the semester, we plan to also experiment with stochastic expectation maximization (see §3.1.1 for details).