Depression is a serious mental disorder that affects many individuals across the globe. Depression (unipolar or bipolar) is characterized by a high rate of relapse or recurrence where a person might experience depressive episodes after non-depressive ones. The symptom patterns for recurrent depressive episodes have not been properly analyzed. Thus, there is a pressing need for systems which can monitor the mental health of individuals at risk to detect initial signs of relapse and recurrence. This points towards an automated system which identifies such signs and facilitates in timely treatment. In this paper, we introduce for the first time a deep learning based prospective monitoring system for the identification of relapse signs using audio-visual cues. The proposed model approximates relapse as the similarity between non-depression and depression samples. Experiments were performed on the DAIC-WOZ dataset and a highest accuracy of 73.21% was obtained using a Siamese network-based approach with one-shot learning regime.