Questions Are All You Need to Train a Dense Passage Retriever

We introduce ART, a new corpus-level au-toencoding approach for training dense retrieval models that does not require any labeled training data. Dense retrieval is a cen-tral challenge for open-domain tasks, such as Open QA, where state-of-the-art methods typically require large supervised datasets with custom hard-negative mining and denoising of positive examples. ART, in contrast, only requires access to unpaired inputs and out-puts (e.g. questions and potential answer doc-uments). It uses a new document-retrieval au-toencoding scheme, where (1) an input question is used to retrieve a set of evidence documents, and (2) the documents are then used to compute the probability of reconstructing the original question. Training for retrieval based on question reconstruction enables effective unsupervised learning of both document and question encoders, which can be later incorpo-rated into complete Open QA systems without any further finetuning. Extensive experiments demonstrate that ART obtains state-of-the-art results on multiple QA retrieval benchmarks with only generic initialization from a pretrained language model, removing the need for labeled data and task-specific losses. 1