Self-Supervision based Task-Specific Image Collection Summarization

Successful applications of deep learning (DL) require a large amount of annotated data. This often restricts the benefits of employing DL to businesses and individuals with large budgets for data-collection and computation. Summarization offers a possible solution by creating much smaller representative datasets that can allow real-time deep learning and analysis of big data and thus democratize the use of DL. In the proposed work, our aim is to explore a novel approach to task-specific image corpus summarization using semantic information and self-supervision. Our method uses a classification-based Wasserstein generative adversarial network (CLSWGAN) as a feature generating network. The model also leverages rotational invariance as selfsupervision and classification on another task. All these objectives are added on features from resnet34 to make it discriminative and robust. The model then generates a summary at inference time by using K-means clustering in the semantic embedding space. Thus, another main advantage of this model is that it does not need to be retrained each time to obtain summaries of different lengths which is an issue with current end-to-end models. We also test our model efficacy by means of rigorous experiments both qualitatively and quantitatively. A. Singh Department of Computer Science Technical University of Munich, Munich E-mail: anuragsingh2@iisc.ac.in *D.K. Sharma (Corresponding Author) Department of Information Technology, Netaji Subhas University of Technology, New Delhi, India E-mail: dk.sharma1982@yahoo.com Department of Computer Science Institute of Information Technology and Management, New Delhi E-mail: sharmashudhir08@gmail.com