Learning in High Risk Situations

In this thesis, we address the age old problem of dealing with scenarios in which the significance of making a mistake while taking a classification decision is very high. We also look deeper into how we can take advantage of deep learning techniques to establish state of the art results and study the representations learnt by the deep reject option classifiers. While supervised techniques have their advantage we explored unsupervised learning in real time scenario of catching malicious traders in Stock Exchanges which are establishments where significance of mistakes are very high. Firstly, we propose deep architectures for learning binary instance specific abstain (reject option) classifiers. To show the effectiveness of the proposed approach, we experiment with several real world datasets. We observe that the proposed approach not only performs comparable to the state-of-the-art approaches, it is also robust against label noise. We also provide visualizations to observe the important features learned by the network corresponding to the abstaining decision. Secondly, we look at Stock exchanges. Due to their prominence, stock exchanges are prone to a variety of attacks. Different types of proprietary fraudulent activity detectors are deployed by stock exchanges to analyze the time series data of trader’s activities or the activity of a particular stock to flag potentially malicious transactions while human analysts probe the flagged transactions further. The key issue faced here is that while the number of anomalous transactions identified can run into thousands or tens of thousands, the number of such transactions that can realistically be probed by human analysts would be a small fraction due to resource constraints. The issue therefore reduces to a dynamic resource allocation problem wherein alerts that represent the most malicious transactions need to be mapped to human analysts for further probing across different time intervals. Thirdly, we develop a multiclass deep learning solution with a rejection option. While binary solutions find relevance in scenarios such as cancer detection where ”yes” or ”no” could be the probable options. In scenarios such as autonomous driving, multiple options could be applicable at each instant. So, we propose a multiclass double ramp loss function. We study the properties of the loss function and establish state of the art results on real world datasets.