Human Machine Joint Decision Making in Distorted Surveillance Scenario

There is plenty of human-machine joint decision-making scenarios in the real world applications, such as driving assistant, suspect identification, medical diagnosis, etc. Existing algorithms propose that machine should give a rejection option when having a high risk or uncertainty score so that the input can be passed to human to make the decision. This is an interesting algorithmic model of human-machine collaboration, but implicitly assumes that humans are more trustworthy than machines. Such an assumption ignores the bias and inconsistency of human, especially in scenarios where machines have superior recognition ability than humans. In this work, we investigate the human-machine joint decision-making problem in distorted surveillance videos, where machines experimentally prove to be comparable to human beings in tolerance to distortion, sometimes even stronger. We propose a new human-machine joint decision-making framework by considering both the confidences of machine and human. To obtain the confidence of human, we build a real-life human decision-making database and propose a deep neural network to estimate human's confidence. Then, confidence alignment method and decision rule are proposed to further output the final decision. Experiments demonstrate that the proposed framework can make less human intervention and more accurate decisions in several human-machine joint decision-making scenarios.